R einbold et al. (2) reported performance results from a study comparing a commercial competitive enzyme-linked immunosorbent assay (cELISA) against a duplex real-time quantitative reverse transcriptase PCR (qRT-PCR) developed for the detection of Anaplasma marginale and A. phagocytophilum 16S rRNA in plasma-free bovine peripheral blood samples.Although estimates of sensitivity and specificity (the ability of a test to accurately detect diseased [or infected] and nondiseased [noninfected] animals, respectively) of the cELISA relative to the qRT-PCR are properly reported in the abstract, they seem to have been confused in the Results and Discussion. For example, in the receiver operating characteristic (ROC) curve analysis in Fig. 2a, the x axis (and corresponding legend) is indicated to represent the true-negative rate (i.e., the specificity), while it should rather be the false-positive rate or 1 Ϫ specificity. In addition, the legend to Fig. 2b incorrectly identifies the two curves: the sensitivity plot is the one represented by diamonds (not squares), while the specificity plot is the one represented by squares (not diamonds).In ROC curve analysis, the optimal cutoff is the one that maximizes the sum of sensitivity and specificity (1). The 15.3% inhibition negative cutoff proposed by the authors was determined by the intersection of the plots of sensitivity and specificity, but it does not optimize sensitivity and specificity (as can be seen in Fig. 2b, using a 20% cutoff yields a much higher specificity while only slightly decreasing sensitivity). The use of an ROC statistic such as Youden's J, would have been appropriate. The sensitivity and specificity reported in Results for the 15.3% cutoff (74.2 and 81.2%) do not agree with Fig. 2b. Since they were estimated by the intersection of the two curves, the sensitivity and specificity values should have been equal (approximately 75%).It also appears that sensitivity and specificity were inverted in the Discussion. The authors compared their results to those of the licensure study for the cELISA, where a sensitivity of 95% was reported with a negative cutoff value set at 28% (3). The authors indicate that "In order to achieve similar sensitivity results, the percentage of inhibition used as the negative cutoff value in our study would have needed to be set at 40%." Without the actual data on hand, we are not in a position to determine exactly what the negative cutoff value should have been to obtain a sensitivity of 95%, but by looking at Fig. 2b, it appears the cutoff should have been in the 2 to 3% range instead. Also in the Discussion, at the 40% inhibition cutoff, it is indicated that sensitivity and specificity estimates are 100% and 60%, respectively. Again, it appears the two terms were confused, since it is the specificity that would be 100% and the sensitivity 60% at this cutoff.The consequence of this confusion is that for various cutoff values, the estimates of sensitivity and specificity reported are erroneous.
REFERENCES1. Dohoo I, Martin W, Stry...