The accuracy of an electronic Pulmonary Embolism Severity Index auto-populated from the electronic health record

Abstract

KeywordsClinical decision support systems; electronic health record; risk assessment; pulmonary embolism; emergency medicine; data completeness Summary Background: The Pulmonary Embolism (PE) Severity Index identifies emergency department (ED) patients with acute PE that can be safely managed without hospitalization. However, the Index comprises 11 weighted variables, complexity that can impede its integration into contextual workflow. Objective: We designed a computerized version of the PE Severity Index (e-Index) to automatically extract the required variables from discrete fields in the electronic health record (EHR). We tested the e-Index on the study population to determine its accuracy compared with a gold standard generated by physician abstraction of the EHR on manual chart review. Methods: This retrospective cohort study included adults with objectively-confirmed acute PE in four community EDs from 2010-2012. Outcomes included performance characteristics of the e-Index for individual values, the number of cases requiring physician editing, and the accuracy of the e-Index risk category (low vs. higher). Results: For the 593 eligible patients, there were 6,523 values automatically extracted. Fifty one of these needed physician editing, yielding an accuracy at the value-level of 99.2% (95% confidence interval [CI], 99.0%-99.4%). Sensitivity was 96.9% (95% CI, 96.0%-97.9%) and specificity was 99.8% (95% CI, 99.7%-99.9%). The 51 corrected values were distributed among 47 cases: 43 cases required the correction of one variable and four cases required the correction of two. At the riskcategory level, the e-Index had an accuracy of 96.8% (95% CI, 95.0%-98.0%), under-classifying 16 higher-risk cases (2.7%) and over-classifying 3 low-risk cases (0.5%). Conclusion: Our automated extraction of variables from the EHR for the e-Index demonstrates substantial accuracy, requiring a minimum of physician editing. This should increase user acceptability and implementation success of a computerized clinical decision support system built around the e-Index, and may serve as a model to automate other complex risk stratification instruments.

Thromboprophylaxis for Patients with High-risk Atrial Fibrillation and Flutter Discharged from the Emergency Department

“…Electronic clinical decision support tools have had success in the ED setting when combined with a strong promotional program and could be readily adapted for use in patients with AF/FL. 83 – 86 A multidisciplinary team at the University of British Columbia designed such an electronic clinical care pathway for ED patients with uncomplicated AF/FL. 48 The pathway included a care map, decision aids, medication orders, management suggestions, and electronic consultation or referral documents, embedded in the computerized physician order entry and integrated electronic health record.…”

Evidence-based Clinical Decision Support Systems for the prediction and detection of three disease states in critical care: A systematic literature review

“…In total, 17 studies were retrospective 33, 48– 50, 52– 55, 58– 66 and five were prospective 51, 56, 57, 67, 68 . Of these studies, 12 were single-center 33, 48, 49, 51, 52, 54, 55, 58, 59, 64– 66 and 10 studies were multi-center 50, 53, 56, 57, 60– 63, 67, 68 . Five studies were time-series 48, 52, 55, 56, 64 , 14 studies were case-series 33, 49, 51, 53, 54, 57– 62, 65, 66, 68 , one was case-control 50 and one was case/time series study 63 .…”

“…The smallest sample of 100 patients came from two single-center retrospective studies 48, 66 . Ten studies had sample sizes of 101–1000 33, 49– 53, 57, 63, 67, 68 ; seven studies had sample sizes of 1001–10,000 54, 55, 59, 60, 62, 64, 65 ; and three had sample sizes larger than 10,000 56, 58, 61 . The largest study included more than 50,000 patients admitted to the ED of two centers over a 3-year period 61 .…”

“…Several published studies did not report their in-patient setting. When reported, some evaluated data from different wards 56, 59, 64, 65, 68 , and some included patients admitted only to the ED 53, 54, 61, 63 , the ICU 48, 60, 67 and the surgical ward 33, 51, 55 .…”

“…One study differed from the rest by developing a computer-aided detection (CAD) system to measure the axial diameter of the right and left pulmonary ventricles, aiding in the diagnosis of pulmonary embolisms 49 . Many learning algorithms were concerned with detecting pulmonary embolisms and deep vein thrombosis 53, 54, 58, 59, 64– 67 as well as pneumonia 33, 48, 57, 60– 63 . Three studies developed machine-learning algorithms to detect COPD 50, 56, 69 .…”