Development and validation of a practical machine-learning triage algorithm for the detection of patients in need of critical care in the emergency department

Abstract

Identifying critically ill patients is a key challenge in emergency department (ED) triage. Mis-triage errors are still widespread in triage systems around the world. Here, we present a machine learning system (MLS) to assist ED triage officers better recognize critically ill patients and provide a text-based explanation of the MLS recommendation. To derive the MLS, an existing dataset of 22,272 patient encounters from 2012 to 2019 from our institution’s electronic emergency triage system (EETS) was used for algorithm training and validation. The area under the receiver operating characteristic curve (AUC) was 0.875 ± 0.006 (CI:95%) in retrospective dataset using fivefold cross validation, higher than that of reference model (0.843 ± 0.005 (CI:95%)). In the prospective cohort study, compared to the traditional triage system’s 1.2% mis-triage rate, the mis-triage rate in the MLS-assisted group was 0.9%. This MLS method with a real-time explanation for triage officers was able to lower the mis-triage rate of critically ill ED patients.

A Prehospital Triage System to Detect Traumatic Intracranial Hemorrhage Using Machine Learning Algorithms

“…In recent years, research on clinical event prediction using machine learning algorithms has been actively conducted, and more complicated and reliable classification methods have become possible. 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 Liu et al 21 developed a machine learning triage system that can be used to detect severity of patients’ injuries using basic clinical information. Therefore, we hypothesized that machine learning approaches might make it possible to produce reliable prediction models even when using insufficient information that was collected on the scene.…”

On the computational approaches for supporting triage systems

“…44 In particular, when considering that decisions made by the triage system can result in death (in case of a wrong decision) or the saved life, the legal and ethical implications require that the model be transparent, even to enable medical staff to have counterintuitive choices. We should report that only a few works by Yu et al 43 and Liu et al 28 pay attention to the explainability of their models. Consequently, the development of explainable AI models is a promising area.…”

“…We should report that only a few works by Yu et al 43 . and Liu et al 28 . pay attention to the explainability of their models.…”

“…Liu et al 28 consider a broad scope to the previous approach. They present an ML algorithm to identify high critical ill patients who have been triaged into level 3 and cannot wait anymore to be admitted to urgent care.…”

Precision Medicine in Emergency Medicine

“…Research on the topic is not new with various kind of focus-from forecasting models for Emergency Department (ED) revisits to triaging anaphylaxis [7]. Such triage system has also been proposed for paediatric populations with good results [11], or for prioritising those patients in ED that will probably need admission to Intensive care Unit (ICU) [12].…”