Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low-prevalence setting

Abstract

Objectives When SARS-CoV-2 prevalence is low, many positive test results are false positives. Confirmatory testing reduces overdiagnosis and nosocomial infection and enables real-world estimates of test specificity and positive predictive value. This study estimates these parameters to evaluate the impact of confirmatory testing, and to improve clinical diagnosis, epidemiological estimation and interpretation of vaccine trials. Methods Over one month, we took all respiratory samples from our laboratory with a patient’s first detection of SARS-CoV-2 RNA (Hologic Aptima SARS-CoV-2 assay or in-house RT-PCR platform), and repeated testing using two platforms. Samples were categorised by source, and by whether clinical details suggested COVID-19 or corroborative testing from another laboratory. We estimated specificity and positive predictive value using maximum likelihood-based approaches. Results Of 19,597 samples, SARS-CoV-2 RNA was detected in 107. 52 corresponded to first-time detection (0.27% of tests on samples without previous detection); further testing detected SARS-CoV-2 RNA ≥1 time (“confirmed”) in 29 (56%), and failed to detect SARS-CoV-2 RNA (“not confirmed”) in 23 (44%). Depending upon assumed parameters, point estimates for specificity and positive predictive value were 99.91%–99.98% and 61.8%–89.8% respectively using the Hologic Aptima SARS-CoV-2 assay, and 97.4%–99.1% and 20.1%–73.8% respectively using an in-house assay. Conclusions Nucleic acid amplification testing for SARS-CoV-2 is highly specific. Nevertheless, when prevalence is low a significant proportion of initially positive results fail to confirm and confirmatory testing substantially reduces false positive detections. Omitting additional testing in samples with higher prior detection probabilities focuses testing where clinically impactful and minimises delay.

Diagnostic tool or screening programme? Asymptomatic testing for SARS-CoV-2 needs clear goals and protocols

“…In many ways, therefore, the test's specificity (how well the test correctly excludes those without infection) matters more in largely-asymptomatic populations: when the prevalence of infection is low, even a highly specific test results in many of the positive results – perhaps even the majority – coming from those without infection (false positives), reflecting the preponderance of individuals in that population without infection. Just as with sensitivity, the lack of gold standard makes quantifying the specificity of a SARS-CoV-2 diagnostic test difficult, but we have shown that when the prevalence of infection is low it is possible to make reliable estimates [29] . The issue of positive tests in those without infection becomes prominent for any test when population prevalence is sufficiently low, but with realistic estimates of a test sensitivity of 70% [22] (note that most of the loss of sensitivity comes at time of sampling, not during laboratory testing) and a test specificity of 99.95% [29] , it is probable that during the summer of 2020, the United Kingdom reached a point where reported SARS-CoV-2 positivity rates mostly represented false positive tests, with week-to-week variations largely representing natural fluctuations in false positive rates ( Fig.…”

Streamlining SARS-CoV-2 confirmatory testing to reduce false positive results

“…In preparation for the second wave of COVID-19, we aimed to streamline the process in our laboratory by analysing our results to identify those most and least likely to require confirmation. Here we report our findings, expand on our previously published work [ 1 ] (which included just the first month’s data), and outline methods that other laboratories can follow to improve their confirmatory testing pathways.…”

“…Our previously published work [ 1 ] shows how false positive results can occur in SARS-CoV-2 testing. Assuming our estimate of 99.9 % specificity [ 1 ], without confirmatory testing, the current count of 300,000 tests per day in the UK [ 2 ] could lead to over 100,000 false positive results a year. Confirmatory testing is crucial to prevent this.…”

Partial N Gene Sequencing for SARS-CoV-2 Verification and Pathway Tracing

“…For example, using two different nucleic acid assays to re-test 52 first-time SARS-CoV-2-positive samples, Skittrall et al detected SARS-CoV-2 RNA in 29 (56%), but not in 23 (44%) of the 52 samples in the second-round testing for confirmation of true SARS-CoV-2 infection. 10 It is of course questionable if the first positive tests, or the repeat negative tests on some of these samples with discordant results were correct. Elimination or minimization of false-positive SARS-CoV-2 RNA test results will reduce unnecessary anxiety among the population and prevent false-positive test results from shutting down schools and workplaces unnecessarily as businesses try to resume normal operations in the community.…”