FACSCount CD4 and Pima CD4 T-Cell Count Systems
FACSCount CD4 and Pima CD4 T-Cell Count Systems
Whereas previous studies assessed the performance of FACSCount CD4 and Pima CD4, our study is the first independent study conducted by the WHO to evaluate the operational characteristics of these systems together with their performance, including error rates.
The FACSCount CD4 system provides absolute CD4 counts up to 5000 cells per microliter and associated CD4% rendering this system particularly suitable for monitoring pediatric HIV-infected patients. The FACSCount CD4 showed very good intra-assay CVs similar to the CVs of the instrumental variation, suggesting that the variability induced by the operator on FACSCount CD4 results is limited. The FACSCount CD4 was compared with the Double-Platform FACSCan and FACSCount in Thailand, but our study is the first to compare it with the SP FACSCalibur. Our results are in agreement with those reported by Pattanapanyasat et al who showed an R >0.97, an absolute bias (LOA) of 3.39 (−52.5 to 59.3) cells per microliter, and a mean similarity of 98.2%.
The Pima CD4 is a battery-powered imaging device that only provides absolute CD4 counts from either capillary or venous blood using disposable cartridges containing lyophilized mAbs. The use of the Pima CD4 system reduces the loss of patients from follow-up, thanks to the short turnaround time for providing CD4 results (20 min). Thus, the physician can immediately take the adequate decision to start or switch ART regimen. The use of Pima CD4 system eliminates the need of a cold chain and air conditioning (except in very hot areas), as its disposable cartridges can be stored up to 30° C and the instrument can be operated up to 40° C. The Pima CD4 showed an intra-assay variation with mean %CV equal or larger than the recommended value of 10% (or 15% in low CD4 counts). This could be partly because of the fact that the precision was measured on 5 different instruments, adding an extra inter-instrument variability to the intra-assay variability itself as was shown in Antwerp (CV = 9.4%). In addition, the Pima CD4 showed intra-assay CVs higher than the instrumental CVs, suggesting that operator added a significant variability. This is in line with the results of previous studies that showed excellent instrumental precision with mean %CV of <5% reflecting the significant contribution of operators to the %CV. The Pima CD4 results using venous blood showed acceptable correlation and agreement with the FACSCalibur results, similar to those reported by previous studies and partially in agreement with the study by Mwau et al in Kenya, comparing Pima CD4 with FACSCount. Mwau et al obtained discordant results comparing Pima CD4 with FACSCalibur but concordant results comparing Pima CD4 with FACSCount. This discrepancy could be because of less reliable results generated by the FACSCalibur used, which showed a substantial absolute bias (LOA) against the FACSCount of −76.5 cells per microliter (−316 to 163). Using capillary blood, the Pima CD4 showed good correlation and acceptable agreement with the FACSCalibur similar to those reported by previous studies. The number of operators (6 in Dar es Salam, against 1 operator for venous blood and 3 nurses performing finger prick in Antwerp) may explain the performance differences observed between the 2 study sites, as shown in previous studies.
Looking at screening of patients in need of treatment, rates of misclassification are in agreement with those reported in previous studies, with a better identification of patients based using venous blood instead of capillary blood on the Pima CD4, even though other studies reported higher misclassification rates using capillary blood. A higher threshold for ART initiation increased the number of misclassified patients. However, the consequences for patients who were upward misclassified at 350 or 500 cells per microliter will be less severe than for patients misclassified upward at 200 cells per microliter. In practice, this delay might be of little importance when the decision is postponed for one visit or in settings where the clinicians take the CD4 count decrease into account instead of the CD4 threshold or even other parameters. Fortunately, when relying on the FACSCount CD4 or the Pima CD4 results rather than on the FACSCalibur results, the proportion of patients for whom the ART initiation would be delayed is much smaller than the proportion of patients who would have been treated too early. This early initiation will improve the well-being of patients and consequently reduce their risk of transmitting HIV. Even if this means that the national programs would have to spend more money to ART, early treatment reduces the risk of opportunistic infections. The only drawback could be that upon ART shortage, some patients who do not require immediate treatment are treated at the expense of patients who require ART more urgently.
Besides the misclassification because of poor instrument precision, physicians need to take into account physiological variation of the CD4 T-cell counts, which also might lead to misclassification. During follow-up of individual HIV-infected patients, it is highly recommended to collect blood for routine CD4 counting at the same time (either in the morning or in the afternoon) to avoid bias between 2 consecutive measurements because of diurnal variations.
The failure rates (percentage of samples where no results were generated) reported in our study on Pima CD4, either on venous or capillary blood, were in agreement with those in previous studies. The occurrence of errors may partially be explained by the lack of experience of personnel who performs finger stick (eg, Pima CD4) and partially by samples that fail to meet the internal quality acceptance criteria implemented by the instrument software itself (eg, Pima CD4 and FACSCount CD4). Errors will obviously increase the cost per test, and repeating a finger prick may not be as simple as repeating a test using venous blood.
In Antwerp, nurses occasionally experienced difficulties to obtain a good capillary blood drop especially without squeezing the finger, which may dilute a blood sample. Alternatively, other patients experienced rather heavy finger bleeding after the finger prick, resulting in either incorrect filling of the cartridges or blood dripping which increased the risks of exposure of blood to the nurses who performed the finger prick. All these aspects make the finger prick more difficult to perform than originally expected, in particular because the training skills required for CD4 counting are more critical than, for instance, for malaria or diabetes finger-prick testing.
The operators experienced the FACSCount CD4 as a simple flow cytometer requiring 2–3 days of training. The FACSCount CD4 can run a batch of 20 samples in 90 minutes and up to 100 samples a day. The instrument operators at the 2 study sites considered the Pima CD4 as a very simple system requiring only one day for training, no need for reagent preparation or extra equipment (except a pipet for pipetting venous blood in the cartridge) to perform the assay. There is no need for a cold chain, and the instrument can be moved from one laboratory to another without the need of instrument recalibration, which is an asset when used in mobile units or for itinerant quality control programs. The Pima CD4 is suitable for a limited number of samples with a maximum daily throughput of 15 samples per instrument (See Table, SDC 3, http://links.lww.com/QAI/A533, which summarizes the characteristics of FACSCount CD4 and Pima CD4 systems).
This study was conducted in 2 reference laboratories with highly skilled personnel who received appropriate training before the start of the study. In both laboratories, the FACSCalibur was used as reference method and both instruments were monitored by external quality assessment programs (QASI means Quality Assessment and Standardization for Immunological measures relevant to HIV/AIDS and UK NEQAS means United Kingdom National External Quality Assessment Service). The 2 sites performed daily calibration of the instruments and ran blood control samples to test the accuracy of the systems. The lack of experience of nurses in Antwerp in performing finger prick for CD4 counting, the different study population at the 2 sites, and the different intra-assay variability method implemented at the 2 sites for the Pima CD4 were noticed as limitations of the study.
In conclusion, the FACSCount CD4 provides reliable absolute CD4 counts and CD4%, which are in excellent agreement with the results obtained on the FACSCalibur as CD4 counting reference method. The FACSCount CD4 is suitable for monitoring HIV-infected adults and children. The results of the Pima CD4 are in acceptable agreement with the FACSCalibur results using either capillary or venous blood. This instrument, which only provides absolute CD4 counts, is primarily suitable for screening adult HIV patients for eligibility to initiate ART in resource-poor settings. Although the Pima CD4 showed a higher misclassification probability than the FACSCount CD4, its high mobility (light and battery powered) and its independence of a cold chain make this instrument a very attractive POC CD4 device, which will increase access to fast CD4 results in more remote areas.
Discussion
Whereas previous studies assessed the performance of FACSCount CD4 and Pima CD4, our study is the first independent study conducted by the WHO to evaluate the operational characteristics of these systems together with their performance, including error rates.
The FACSCount CD4 system provides absolute CD4 counts up to 5000 cells per microliter and associated CD4% rendering this system particularly suitable for monitoring pediatric HIV-infected patients. The FACSCount CD4 showed very good intra-assay CVs similar to the CVs of the instrumental variation, suggesting that the variability induced by the operator on FACSCount CD4 results is limited. The FACSCount CD4 was compared with the Double-Platform FACSCan and FACSCount in Thailand, but our study is the first to compare it with the SP FACSCalibur. Our results are in agreement with those reported by Pattanapanyasat et al who showed an R >0.97, an absolute bias (LOA) of 3.39 (−52.5 to 59.3) cells per microliter, and a mean similarity of 98.2%.
The Pima CD4 is a battery-powered imaging device that only provides absolute CD4 counts from either capillary or venous blood using disposable cartridges containing lyophilized mAbs. The use of the Pima CD4 system reduces the loss of patients from follow-up, thanks to the short turnaround time for providing CD4 results (20 min). Thus, the physician can immediately take the adequate decision to start or switch ART regimen. The use of Pima CD4 system eliminates the need of a cold chain and air conditioning (except in very hot areas), as its disposable cartridges can be stored up to 30° C and the instrument can be operated up to 40° C. The Pima CD4 showed an intra-assay variation with mean %CV equal or larger than the recommended value of 10% (or 15% in low CD4 counts). This could be partly because of the fact that the precision was measured on 5 different instruments, adding an extra inter-instrument variability to the intra-assay variability itself as was shown in Antwerp (CV = 9.4%). In addition, the Pima CD4 showed intra-assay CVs higher than the instrumental CVs, suggesting that operator added a significant variability. This is in line with the results of previous studies that showed excellent instrumental precision with mean %CV of <5% reflecting the significant contribution of operators to the %CV. The Pima CD4 results using venous blood showed acceptable correlation and agreement with the FACSCalibur results, similar to those reported by previous studies and partially in agreement with the study by Mwau et al in Kenya, comparing Pima CD4 with FACSCount. Mwau et al obtained discordant results comparing Pima CD4 with FACSCalibur but concordant results comparing Pima CD4 with FACSCount. This discrepancy could be because of less reliable results generated by the FACSCalibur used, which showed a substantial absolute bias (LOA) against the FACSCount of −76.5 cells per microliter (−316 to 163). Using capillary blood, the Pima CD4 showed good correlation and acceptable agreement with the FACSCalibur similar to those reported by previous studies. The number of operators (6 in Dar es Salam, against 1 operator for venous blood and 3 nurses performing finger prick in Antwerp) may explain the performance differences observed between the 2 study sites, as shown in previous studies.
Looking at screening of patients in need of treatment, rates of misclassification are in agreement with those reported in previous studies, with a better identification of patients based using venous blood instead of capillary blood on the Pima CD4, even though other studies reported higher misclassification rates using capillary blood. A higher threshold for ART initiation increased the number of misclassified patients. However, the consequences for patients who were upward misclassified at 350 or 500 cells per microliter will be less severe than for patients misclassified upward at 200 cells per microliter. In practice, this delay might be of little importance when the decision is postponed for one visit or in settings where the clinicians take the CD4 count decrease into account instead of the CD4 threshold or even other parameters. Fortunately, when relying on the FACSCount CD4 or the Pima CD4 results rather than on the FACSCalibur results, the proportion of patients for whom the ART initiation would be delayed is much smaller than the proportion of patients who would have been treated too early. This early initiation will improve the well-being of patients and consequently reduce their risk of transmitting HIV. Even if this means that the national programs would have to spend more money to ART, early treatment reduces the risk of opportunistic infections. The only drawback could be that upon ART shortage, some patients who do not require immediate treatment are treated at the expense of patients who require ART more urgently.
Besides the misclassification because of poor instrument precision, physicians need to take into account physiological variation of the CD4 T-cell counts, which also might lead to misclassification. During follow-up of individual HIV-infected patients, it is highly recommended to collect blood for routine CD4 counting at the same time (either in the morning or in the afternoon) to avoid bias between 2 consecutive measurements because of diurnal variations.
The failure rates (percentage of samples where no results were generated) reported in our study on Pima CD4, either on venous or capillary blood, were in agreement with those in previous studies. The occurrence of errors may partially be explained by the lack of experience of personnel who performs finger stick (eg, Pima CD4) and partially by samples that fail to meet the internal quality acceptance criteria implemented by the instrument software itself (eg, Pima CD4 and FACSCount CD4). Errors will obviously increase the cost per test, and repeating a finger prick may not be as simple as repeating a test using venous blood.
In Antwerp, nurses occasionally experienced difficulties to obtain a good capillary blood drop especially without squeezing the finger, which may dilute a blood sample. Alternatively, other patients experienced rather heavy finger bleeding after the finger prick, resulting in either incorrect filling of the cartridges or blood dripping which increased the risks of exposure of blood to the nurses who performed the finger prick. All these aspects make the finger prick more difficult to perform than originally expected, in particular because the training skills required for CD4 counting are more critical than, for instance, for malaria or diabetes finger-prick testing.
The operators experienced the FACSCount CD4 as a simple flow cytometer requiring 2–3 days of training. The FACSCount CD4 can run a batch of 20 samples in 90 minutes and up to 100 samples a day. The instrument operators at the 2 study sites considered the Pima CD4 as a very simple system requiring only one day for training, no need for reagent preparation or extra equipment (except a pipet for pipetting venous blood in the cartridge) to perform the assay. There is no need for a cold chain, and the instrument can be moved from one laboratory to another without the need of instrument recalibration, which is an asset when used in mobile units or for itinerant quality control programs. The Pima CD4 is suitable for a limited number of samples with a maximum daily throughput of 15 samples per instrument (See Table, SDC 3, http://links.lww.com/QAI/A533, which summarizes the characteristics of FACSCount CD4 and Pima CD4 systems).
This study was conducted in 2 reference laboratories with highly skilled personnel who received appropriate training before the start of the study. In both laboratories, the FACSCalibur was used as reference method and both instruments were monitored by external quality assessment programs (QASI means Quality Assessment and Standardization for Immunological measures relevant to HIV/AIDS and UK NEQAS means United Kingdom National External Quality Assessment Service). The 2 sites performed daily calibration of the instruments and ran blood control samples to test the accuracy of the systems. The lack of experience of nurses in Antwerp in performing finger prick for CD4 counting, the different study population at the 2 sites, and the different intra-assay variability method implemented at the 2 sites for the Pima CD4 were noticed as limitations of the study.
In conclusion, the FACSCount CD4 provides reliable absolute CD4 counts and CD4%, which are in excellent agreement with the results obtained on the FACSCalibur as CD4 counting reference method. The FACSCount CD4 is suitable for monitoring HIV-infected adults and children. The results of the Pima CD4 are in acceptable agreement with the FACSCalibur results using either capillary or venous blood. This instrument, which only provides absolute CD4 counts, is primarily suitable for screening adult HIV patients for eligibility to initiate ART in resource-poor settings. Although the Pima CD4 showed a higher misclassification probability than the FACSCount CD4, its high mobility (light and battery powered) and its independence of a cold chain make this instrument a very attractive POC CD4 device, which will increase access to fast CD4 results in more remote areas.
