Sigma Metrics for Assessing Accuracy of Molecular Testing

Multicenter Evaluation of Two Next-Generation HIV-1 Quantitation Assays, Aptima Quant Dx and Cobas 6800, in Comparison to the RealTi m e HIV-1 Reference Assay

“… Sigma precision analysis for all laboratories and all assays. Plots were performed according to Westgard and Lucic ( 8 ) for each participating laboratory, subtype, and dilution. Diagonal lines on the chart represent different sigma zones, from world-class-quality performance (more than 6σ) near the graph's origin, to excellent performance (5σ), good performance (4σ), etc., with the upper right quadrant of the graph representing less than 2σ performance, which is considered unacceptable, unstable, and unreliable.…”

“…As HIV assays exhibit various degree of variability at low VLs around the clinically relevant cutoffs, assay performance at these key thresholds can be objectively evaluated using the sigma metrics ( 8 ). Westgard and Lucic ( 8 ) used previously published data on HIV VL assays ( 6 , 26 ) to retrospectively calculate the sigma metrics for two comparator assays and found the RealTi m e assay to be 6σ (world-class; in both studies) and the CAP/CTM assay to be 4σ (good in reference 6 ) to 3σ (marginal in reference 26 ). The Westgard sigma analysis for RealTi m e is in agreement with our findings.…”

“…A sigma analysis of imprecision was also conducted, as described by Westgard and Lucic ( 8 ), for quality controls in which the sigma metric is calculated using the following equation: sigma metric = (TEa − bias)/%CV, where TEa is the allowable total error and CV is the coefficient of variation. The %CV values were calculated from nonlogarithmic values.…”

“…Consequently, current HIV-1 VL assay systems need to deliver precise, reproducible, specific, and sensitive results, particularly around the clinical relevant benchmarks (50 and 200 copies/ml). Because patient management decisions rely entirely on each test result, the reliability of the HIV VL assay used in the clinical laboratory should be assessed and confirmed through objective rigorous modern analysis methods, such as the sigma metrics ( 8 ). The Greek letter sigma is used to represent standard deviations and to show how wide the distribution of values spans.…”

“…Therefore, it is recommended that laboratories estimate the total analytical error by combining the estimate of bias from a method comparison study and the estimate of precision from a replication study. Accordingly, we used a multiple of the standard deviation (SD) or coefficient of variation (CV) and total analytical error/bias for a 95% confidence interval, as recently published ( 8 ).…”