Evaluating the accuracy of data extracted from electronic health records into MedicineInsight, a national Australian general practice database

Abstract

IntroductionMedicineInsight is a database containing de-identified electronic health records (EHRs) from over 700 Australian general practices. Previous research validated algorithms used to derive medical condition flags in MedicineInsight, but the accuracy of data fields following EHR extractions from clinical practices and data warehouse transformation processes have not been formally validated. ObjectivesTo examine the accuracy of the extraction and transformation of EHR fields for selected demographics, observations, diagnoses, prescriptions, and tests into MedicineInsight. MethodsWe benchmarked MedicineInsight values against those recorded in original EHRs. Forty-six general practices contributing data to MedicineInsight met our eligibility criteria, eight were randomly selected, and four agreed to participate. We randomly selected 200 patients ≥ 18 years of age within each participating practice from MedicineInsight. Trained staff reviewed the original EHRs for the selected patients and recorded data from the relevant fields. We calculated the percentage of agreement (POA) between MedicineInsight and EHR data for all fields; Cohen's Kappa for categorical and intra-class correlation (ICC) for continuous measures; and sensitivity, specificity, and positive and negative predictive values (PPV/NPV) for diagnoses. ResultsA total of 796 patients were included in our analysis. All demographic characteristics, observations, diagnoses, prescriptions and random pathology test results had excellent (> 90%) POA, Kappa, and ICC. POA for most recent pathology/imaging test was moderate (81%, [95% CI: 78% to 84%]). Sensitivity, specificity, PPV, and NPV were excellent (> 90%) for all but one of the examined diagnoses which had a poor PPV. ConclusionsOverall, our study shows good agreement between the majority of MedicineInsight data and those from original EHRs, suggesting MedicineInsight data extraction and warehousing procedures accurately conserve the data in these key fields. Discrepancies between test data may have arisen due to how data from pathology, radiology and other imaging providers are stored in EHRs and MedicineInsight and this requires further investigation.

The Australian Health Care Homes trial: quality of care and patient outcomes. A propensity score‐matched cohort study

“…However, information on differences between those who did and did not enrol was not available to us. Practice data extracts accurately reflect the data in the source electronic medical records, 24 but the completeness and validity of these data may differ between practices, and we found data harmonisation a problem. Further, the practice data extracts and linked data were not integrated into a single dataset, precluding the inclusion of all variables for propensity score matching and assessment of the relationships between measures of primary care and other health outcomes at the individual level.…”

Data Quality–Driven Improvement in Health Care: Systematic Literature Review

“…The most common method to assess accuracy in healthcare involves the comparison of EHR data to a reference gold standard, which may include paper records, manual data reviews, triangulation of data from multiple sources, or interviews with patients [22]. Measurement of data accuracy can identify issues such as lack of specificity or precision [33]. Previous work found that code precision can be related to staff training and/or use of multiple EHR systems [33][34][35][36].…”

“…Measurement of data accuracy can identify issues such as lack of specificity or precision [33]. Previous work found that code precision can be related to staff training and/or use of multiple EHR systems [33][34][35][36]. As Cook et al [37] noted in a review of DQ issues affecting social determinants data, imprecise codified data may affect minority groups disproportionately, which in turn may affect secondary research outcomes.…”