BackgroundWith the recent implementation of the Medicare Quality Payment Program, providers face increasing accountability for delivering high‐quality care. Such pay‐for‐performance programs aim to leverage systematic data captured by electronic health record (EHR) systems to measure performance; however, the fidelity of EHR query for assessing performance has not been validated compared with manual chart review. We sought to determine whether our institution's methodology of EHR query could accurately identify cases in which providers failed to prescribe statins for eligible patients with coronary artery disease.Methods and ResultsA total of 9459 patients with coronary artery disease were seen at least twice at the Emory Clinic between July 2014 and June 2015, of whom 1338 (14.1%, 95% confidence interval 13.5–14.9%) had no statin prescription or exemption per EHR query. A total of 120 patient cases were randomly selected and reviewed by 2 physicians for further adjudication. Of the 120 cases initially classified as statin prescription failures, only 21 (17.5%; 95% confidence interval, 11.7–25.3%) represented true failure following physician review.ConclusionsSole reliance on EHR data query to measure quality metrics may lead to significant errors in assessing provider performance. Institutions should be cognizant of these potential sources of error, provide support to medical providers, and form collaborative data management teams to promote and improve meaningful use of EHRs. We propose actionable steps to improve the accuracy of EHR data query that require hypothesis testing and prospective validation in future studies.

Background: Programs such as Meaningful Use leverage electronic health record (EHR) systems to hold medical providers accountable for delivering high-quality care. However, the fidelity of querying population-level EHR data for assessing quality of care compared with manual chart review is unclear. Objective: To determine the accuracy of EHR data query in measuring the rate of statin prescriptions in eligible patients with coronary artery disease (CAD). Methods: The study was performed at The Emory Clinic (TEC), a large, academic, multispecialty clinic practice in Atlanta, GA. Query of TEC’s clinical data warehouse which captures data from TEC’s EHR system identified 29,713 total outpatient encounters for CAD between July 1, 2014 and June 30, 2015, of which 3,839 encounters (13%) had no detectable statin prescription or documented exemption. Of these 3,839 “failure” cases, 120 patient charts were randomly subjected to manual adjudication by two physicians and were categorized as follows: 1) CAD recorded on EHR problem list but without documented evidence of clinical, symptomatic CAD as defined by the 2013 AHA / ACC cholesterol guidelines; 2) Failure of EHR data query to recognize an active statin prescription on the EHR medication list; 3) Failure of EHR data query to recognize a statin allergy or contraindication in a discrete data field (e.g. allergy section or problem list); 4) Statin exemption listed in only free-text clinical notes; 5) Comorbidity that downgrades strength of evidence supporting statin use in CAD (e.g. end-stage renal disease); or 6) True failure of provider to prescribe statin therapy (i.e. true gap in care). Results: Of 120 patient charts reviewed, only 35 (29.2%) represented a true failure of to prescribe statins in eligible CAD patients. Forty (33.3%) patients did not meet criteria for clinical CAD; 21 (17.5%) had statin exemption documented in free-text format only; 17 (14.2%) had statin exemption listed in discrete EHR fields; 6 (5%) were on a statin according to the EHR medication list; and 1 (0.8%) had end-stage renal disease. Conclusions: Among outpatients with presumed CAD who were identified by EHR data query as “failures” to receive statin prescriptions, an estimated fewer than one-third constituted a true gap in care. These findings raise concern about the accuracy of EHR data query in measuring performance in the Meaningful Use era.