Evaluation of Version 4 of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage

Sax, Dana R.; Warton, E. Margaret; Mark, Dustin G.; Vinson, David R.; Kene, Mamata V.; Ballard, Dustin; Vitale, Tina J; McGaughey, Katherine R; Beardsley, Aaron; Pines, Jesse M.; Reed, Mary

doi:10.1001/jamanetworkopen.2023.3404

Cited by 41 publications

(24 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adding triage nurse clinical assessments to the prediction model led to a similar or higher AUC for predicting hospitalization compared to results reported in earlier models. 4 In this study, we found that deep learning methods could achieve high sensitivity at lower hospital admission risk thresholds (>90% in all models at 10% admission threshold) and high specificity at higher hospital risk thresholds (>95% in each model at 30% and 50% risk thresholds).…”

Section: Discussionmentioning

confidence: 53%

“…A large multicenter study estimated the ESI to be accurate in <50% of high acuity cases, with no difference in overall accuracy according to nurse experience 2 . Using novel measures to define under‐ and overtriage for each ESI level, we recently reported that mistriage with ESI occurs in over 30% of encounters, with overtriage much more common than under triage 4 …”

Section: Discussionmentioning

confidence: 99%

“…Although we were not able to directly compare model results in our study with predictive accuracy of the ESI because of distinct prediction targets (the ESI predicts 5 levels of patient acuity and resource needs, whereas our models predict hospital admission and fast‐track eligibility), our findings suggest ML‐models offer an opportunity for improvement. In prior work, we found the sensitivity and specificity of ESI to predict low‐acuity, low resource‐needs patients (correctly assigning an ESI IV or V among patients who used <2 resources and had no critical interventions) were 50.0% and 96.8%, respectively, whereas the sensitivity and specificity of ESI to predict a critically ill patient (defined as correctly assigning an ESI I or II among patients with a critical care intervention) were 65.9% and 83.4%, respectively 4 . In this study, we found that deep learning methods could achieve high sensitivity at lower hospital admission risk thresholds (>90% in all models at 10% admission threshold) and high specificity at higher hospital risk thresholds (>95% in each model at 30% and 50% risk thresholds).…”

Section: Discussionmentioning

confidence: 99%

“…Prior studies have found that the most commonly used triage system in the United States, the 5‐level Emergency Severity Index (ESI), has limited accuracy and reliability 1–6 . The ESI was developed in 1999 3 and is used in over 70% of EDs across the United States 7 .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

et al. 2023

Self Cite

View full text Add to dashboard Cite

ObjectivesEfficient and accurate emergency department (ED) triage is critical to prioritize the sickest patients and manage department flow. We explored the use of electronic health record data and advanced predictive analytics to improve triage performance.MethodsUsing a data set of over 5 million ED encounters of patients 18 years and older across 21 EDs from 2016 to 2020, we derived triage models using deep learning to predict 2 outcomes: hospitalization (primary outcome) and fast‐track eligibility (exploratory outcome), defined as ED discharge with <2 resource types used (eg, laboratory or imaging studies) and no critical events (eg, resuscitative medications use or intensive care unit [ICU] admission). We report area under the receiver operator characteristic curve (AUC) and 95% confidence intervals (CI) for models using (1) triage variables alone (demographics and vital signs), (2) triage nurse clinical assessment alone (unstructured notes), and (3) triage variables plus clinical assessment for each prediction target.ResultsWe found 12.7% of patients were hospitalized (n = 673,659) and 37.0% were fast‐track eligible (n = 1,966,615). The AUC was lowest for models using triage variables alone: AUC 0.77 (95% CI 0.77–0.78) and 0.70 (95% CI 0.70–0.71) for hospitalization and fast‐track eligibility, respectively, and highest for models incorporating clinical assessment with triage variables for both hospitalization and fast‐track eligibility: AUC 0.87 (95% CI 0.87–0.87) for both prediction targets.ConclusionOur findings highlight the potential to use advanced predictive analytics to accurately predict key ED triage outcomes. Predictive accuracy was optimized when clinical assessments were added to models using simple structured variables alone.

show abstract

Section: Discussionmentioning

confidence: 53%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several studies have documented major disparities in the assignment of ESI triage scores to ED patients according to their race and ethnicity . Specifically, Black and Hispanic patients tend to be assigned lower-acuity triage scores than were White patients, even with identical chief symptoms, such as abdominal pain .…”

Section: Introductionmentioning

confidence: 99%

Race and Ethnicity and Primary Language in Emergency Department Triage

Joseph,

Kennedy,

Landry

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceEmergency department (ED) triage substantially affects how long patients wait for care but triage scoring relies on few objective criteria. Prior studies suggest that Black and Hispanic patients receive unequal triage scores, paralleled by disparities in the depth of physician evaluations.ObjectivesTo examine whether racial disparities in triage scores and physician evaluations are present across a multicenter network of academic and community hospitals and evaluate whether patients who do not speak English face similar disparities.Design, Setting, and ParticipantsThis was a cross-sectional, multicenter study examining adults presenting between February 28, 2019, and January 1, 2023, across the Mass General Brigham Integrated Health Care System, encompassing 7 EDs: 2 urban academic hospitals and 5 community hospitals. Analysis included all patients presenting with 1 of 5 common chief symptoms.ExposuresEmergency department nurse-led triage and physician evaluation.Main Outcomes and MeasuresAverage Triage Emergency Severity Index [ESI] score and average visit work relative value units [wRVUs] were compared across symptoms and between individual minority racial and ethnic groups and White patients.ResultsThere were 249 829 visits (149 861 female [60%], American Indian or Alaska Native 0.2%, Asian 3.3%, Black 11.8%, Hispanic 18.8%, Native Hawaiian or Other Pacific Islander &lt;0.1%, White 60.8%, and patients identifying as Other race or ethnicity 5.1%). Median age was 48 (IQR, 29-66) years. White patients had more acute ESI scores than Hispanic or Other patients across all symptoms (eg, chest pain: Hispanic, 2.68 [95% CI, 2.67-2.69]; White, 2.55 [95% CI, 2.55-2.56]; Other, 2.66 [95% CI, 2.64-2.68]; P &lt; .001) and Black patients across most symptoms (nausea/vomiting: Black, 2.97 [95% CI, 2.96-2.99]; White: 2.90 [95% CI, 2.89-2.91]; P &lt; .001). These differences were reversed for wRVUs (chest pain: Black, 4.32 [95% CI, 4.25-4.39]; Hispanic, 4.13 [95% CI, 4.08-4.18]; White 3.55 [95% CI, 3.52-3.58]; Other 3.96 [95% CI, 3.84-4.08]; P &lt; .001). Similar patterns were seen for patients whose primary language was not English.Conclusions and RelevanceIn this cross-sectional study, patients who identified as Black, Hispanic, and Other race and ethnicity were assigned less acute ESI scores than their White peers despite having received more involved physician workups, suggesting some degree of mistriage. Clinical decision support systems might reduce these disparities but would require careful calibration to avoid replicating bias.

show abstract

Variation in Emergency Department Physician Admitting Practices and Subsequent Mortality

Coussens,

2024

JAMA Intern Med

View full text Add to dashboard Cite

ImportanceAn emergency department (ED) physician’s decision to admit a patient to the hospital plays a pivotal role in determining the type and intensity of care that patient will receive. ED physicians vary widely in their propensity to admit patients to the hospital, but it is unknown whether higher admission propensities result in lower subsequent mortality rates.ObjectiveTo measure the variation in ED physicians’ admission propensities and estimate their association with patients’ subsequent mortality rates.Design, Setting, and ParticipantsThis cross-sectional study used nationwide Veterans Affairs electronic health record data from January 2011 to December 2019, comparing physicians practicing within the same ED. The study population was composed of patients visiting the ED with 1 of the 3 most frequent chief complaints in US EDs (chest pain, shortness of breath, and abdominal pain). The data analyses were performed from May 2022 to October 2024.Main Outcomes and MeasuresThe main outcomes were variation in physicians’ adjusted admission rates, short inpatient stays (&lt;24 hours), and 30-day mortality.ResultsThe study population included 2098 physicians seeing 2 137 681 patient visits across 105 EDs. The mean (SD) patient age was 63 (15) years, and 9.8% of patients were female. The mean admission rate was 41.2%, and the mean 30-day mortality rate was 2.5%. Physicians’ adjusted admission rates varied greatly within the same ED (eg, for chest pain: 90th percentile of physicians, 56.6% admitted vs 10th percentile, 32.6% admitted; difference, 24.0 percentage points), despite finding no association between these adjusted admission rates and patients’ prior health status as measured by their Elixhauser Comorbidity Index score before the ED visit. However, patients admitted by physicians with higher admission rates were more likely to be discharged within 24 hours (eg, 31.0% vs 24.8%, respectively), while patients of physicians with higher admission rates had subsequent mortality rates that were no less than those of patients of physicians with lower admission rates.Conclusions and RelevanceThis cross-sectional study demonstrated that ED physicians vary widely in their admission propensity, despite seeing patients with similar prior health status. The results suggest that patients treated by physicians with higher admission propensities are more likely to be discharged after only a short inpatient stay and experience no reduction in subsequent mortality rates.

show abstract

Evaluation of Version 4 of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage

Cited by 41 publications

References 68 publications

Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis

Race and Ethnicity and Primary Language in Emergency Department Triage

Variation in Emergency Department Physician Admitting Practices and Subsequent Mortality

Contact Info

Product

Resources

About