Evaluating the performance of machine learning methods for risk estimation of delirium in patients hospitalized from the emergency department

Abstract

Introduction Delirium is a cerebral dysfunction seen commonly in the acute care setting. It is associated with increased mortality and morbidity and is frequently missed in the emergency department (ED) and inpatient care by clinical gestalt alone. Identifying those at risk of delirium may help prioritize screening and interventions in the hospital setting. Objective Our objective was to leverage electronic health records to identify a clinically valuable risk estimation model for prevalent delirium in patients being transferred from the ED to inpatient units. Methods This was a retrospective cohort study to develop and validate a risk model to detect delirium using patient data available from prior visits and ED encounter. Electronic health records were extracted for patients hospitalized from the ED between January 1, 2014, and December 31, 2020. Eligible patients were aged 65 or older, admitted to an inpatient unit from the emergency department, and had at least one DOSS assessment or CAM‐ICU recorded within 72 h of hospitalization. Six machine learning models were developed to estimate the risk of delirium using clinical variables including demographic features, physiological measurements, medications administered, lab results, and diagnoses. Results A total of 28,531 patients met the inclusion criteria with 8057 (28.4%) having a positive delirium screening within the outcome observation period. Machine learning models were compared using the area under the receiver operating curve (AUC). The gradient boosted machine achieved the best performance with an AUC of 0.839 (95% CI, 0.837–0.841). At a 90% sensitivity threshold, this model achieved a specificity of 53.5% (95% CI 53.0%–54.0%) a positive predictive value of 43.5% (95% CI 43.2%–43.9%), and a negative predictive value of 93.1% (95% CI 93.1%–93.2%). A random forest model and L1‐penalized logistic regression also demonstrated notable performance with AUCs of 0.837 (95% CI, 0.835–0.838) and 0.831 (95% CI, 0.830–0.833) respectively. Conclusion This study demonstrated the use of machine learning algorithms to identify a combination of variables that enables an estimation of risk of positive delirium screens early in hospitalization to develop prevention or management protocols.

Enhancing healthcare access for an older population: The age‐friendly emergency department

“…Machine learning may be a promising tool to reduce the testing time needed or identify those at higher risk for testing. 43 , 44 , 45 Once identified, there are many ways to manage delirium. AFED should have delirium pathways or toolkits that may utilize additional bedside volunteers or patient care technicians, mobilization with therapists, an evaluation of potential reversible causes of delirium, and reducing agitation from pain or tethering devices such as IVs and cardiac monitors.…”

Beyond confusion: Embedding psychiatry in delirium research and clinical practice

“…Machine learning could provide a data-driven approach to early identification of patients at risk of delirium. Mueller et al 15 evaluated algorithms applied in emergency department records using variables such as demographic features, physiological measurements, medications administered, laboratory results and diagnoses. Setting the threshold for sensitivity at 90%, a gradient-boosted machine learning model achieved a specificity of 53.5%.…”