Positive Predictive Value of COVID-19 ICD-10 Diagnosis Codes Across Calendar Time and Clinical Setting

Abstract

Purpose: To estimate the positive predictive value (PPV) of International Classification of Diseases, Tenth Revision (ICD-10) code U07.1, COVID-19 virus identified, in the Department of Veterans of Affairs (VA). Patients and Methods: Records of ICD-10 code U07.1 from inpatient, outpatient, and emergency/urgent care settings were extracted from VA medical record data from 4/01/2020 to 3/31/2021. A weighted, random sample of 1500 records from each quarter of the one-year observation period was reviewed by study personnel to confirm active COVID-19 infection at the time of diagnosis and classify reasons for false positive records. PPV was estimated overall and compared across clinical setting and quarters. Results: We identified 664,406 records of U07.1. Among the 1500 reviewed, 237 were false positives (PPV: 84.2%, 95% CI: 82.4-86.0). PPV ranged from 77.7% in outpatient settings to 93.8% in inpatient settings and was 83.3% in quarter 1, 80.5% in quarter 2, 86.1% in quarter 3, and 83.6% in quarter 4. The most common reasons for false positive records were history of COVID-19 (44.3%) and orders for laboratory tests (21.5%). Conclusion:The PPV of ICD-10 code U07.1 is low, especially in outpatient settings. Directed training may improve accuracy of coding to levels that are deemed adequate for future use in surveillance efforts.

Real-World Evidence of COVID-19 Patients’ Data Quality in the Electronic Health Records

“…The current infrastructure and complexity of EHR systems vary across hospitals, which limits the capability of using EHR data for research purposes [ 9 ]. Data quality and related issues have been studied in many contexts, and the findings can vary across different institutions and different research studies [ 9 , 25 , 26 , 27 , 28 ]. Many such issues are generated during the documentation process at the point of care [ 28 ].…”

“…In contrast, ICD-10 codes are also known to give low sensitivity even though they are have high specificity [ 28 ]. Lynch et al evaluated the performance of ICD-10 code U07.1 for identifying COVID-19 patients using a manual chart review as a gold standard, and they found that the performance was low [ 26 ]. Similarly, DeLozier et al found that using laboratory testing (sensitivity = 93%) only to define COVID-19 patients outperformed the use of ICD-10 code U07.1 (sensitivity = 46.4%), which can be improved when combining the output of both definitions of ICD-10 and laboratory testing to yield a sensitivity of 100% [ 27 ].…”

“…Similarly, DeLozier et al found that using laboratory testing (sensitivity = 93%) only to define COVID-19 patients outperformed the use of ICD-10 code U07.1 (sensitivity = 46.4%), which can be improved when combining the output of both definitions of ICD-10 and laboratory testing to yield a sensitivity of 100% [ 27 ]. Lynch et al reported the use of ICD-10 codes either alone or supported with laboratory tests is not sufficient for surveillance and research [ 26 ], as ICD-10 codes do not appear to capture cases correctly [ 30 ]. In addition, the absence of a diagnostic code in the EHR does not necessarily represent the absence of the phenotype [ 28 ].…”

“…EHR data might rarely be error-free; therefore, evaluating the quality of EHR data is important for deriving research-grade and computable phenotypes and public health real-time tracking and response [ 9 , 28 , 34 ]. The need for further studies in assessing data quality across different EHR systems has been reported by several studies [ 9 , 25 , 26 , 27 , 28 ]. Ann Marie Navar [ 35 ] provided an example of the COVID-19 data quality issue and stated that “the present example is one of many that show how far we remain from being able to use EHR data alone to conduct reliable, in-depth, and accurate observational research” [ 35 ].…”

“…Ann Marie Navar [ 35 ] provided an example of the COVID-19 data quality issue and stated that “the present example is one of many that show how far we remain from being able to use EHR data alone to conduct reliable, in-depth, and accurate observational research” [ 35 ]. Moreover, we found that most studies focused on only COVID-19 confirmed cases [ 25 , 26 , 27 , 28 ], one study focused on COVID-19 confirmed and susceptible cases [ 15 ], and none of these studies included COVID-19 probable cases [ 15 , 25 , 26 , 27 , 28 ]. Assessing the quality of symptom- and social-history-based definitions, such as COVID-19 susceptible and probable cases, is challenging and requires a manual chart review.…”