Early prediction of hospital admission of emergency department patients

Abstract

Objective: The early prediction of hospital admission is important to ED patient management. Using available electronic data, we aimed to develop a predictive model for hospital admission. Methods: We analysed all presentations to the ED of a tertiary referral centre over 7 years. To our knowledge, our data set of nearly 600 000 presentations is the largest reported. Using demographic, clinical, socioeconomic, triage, vital signs, pathology data and keywords in electronic notes, we trained a machine learning (ML) model with presentations from 2015 to 2020 and evaluated it on a held-out data set from 2021 to mid-2022. We assessed electronic medical records (EMRs) data at patient arrival (baseline), 30, 60, 120 and 240 min after ED presentation. Results:The training data set included 424 354 data points and the validation data set 53 403. We developed and trained a binary classifier to predict inpatient admission. On a held-out test data set of 121 258 data points, we predicted admission with 86% accuracy within 30 min of ED presentation with 94% discrimination. All models for different time points from ED presentation produced an area under the receiver operating characteristic curve (AUC) ≥0.93 for admission overall, with sensitivity/specificity/ F1-scores of 0.83/0.90/0.84 for any inpatient admission at 30 min after presentation and 0.81/0.92/0.84 at baseline. The models retained lower but still high AUC levels when separated for short stay units or inpatient admissions. Conclusion:We combined available electronic data and ML technology to achieve excellent predictive performance for subsequent hospital admission. Such prediction may assist with patient flow.

Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation

“…This is the first study of its kind to prospectively monitor and report an AI system performance in an Australian ED postimplementation. ML models have been developed to assist with different aspects of the triage process including assignment of triage score, 11,12 patient disposition 8,10,13 and identification of critical illness. 14,15 Many of the tools developed to support triage processes have shown clinical utility; however, their development has been largely based on retrospective data that has been processed and cleaned preimplementation.…”

“…We propose that every AI system integrated into clinical practice should undertake post-implementation diagnostic evaluation and regular performance reporting to ensure stable performance to avoid unintentional consequences. The original AI model was designed to only utilise triage notes as an input to determine the prediction, therefore we hypothesise that by retraining the system with other patient oriented data, the diagnostic evaluation will improve, as shown by a recent study conducted by Kishore et al 8 This model shows different accuracy when it comes to predict admission to inpatient units. While the system was able to predict up to 80% of medical admissions, it fails to produce the same performance for admission to cardiology or orthopaedic units.…”

“…We propose that every AI system integrated into clinical practice should undertake post‐implementation diagnostic evaluation and regular performance reporting to ensure stable performance to avoid unintentional consequences. The original AI model was designed to only utilise triage notes as an input to determine the prediction, therefore we hypothesise that by retraining the system with other patient oriented data, the diagnostic evaluation will improve, as shown by a recent study conducted by Kishore et al 8 …”

“…Recently, faster and more powerful computers have brought about an expansion of AI systems and tools in healthcare 7 . While there are many studies evaluating the application of AI and natural language processing (NLP) in emergency medicine, the majority use retrospective data to train and test their algorithms 8 . This means that there are limited prospective studies evaluating the real‐time performance of AI NLP systems and their effectiveness in clinical practice.…”

Attitudes towards artificial intelligence in emergency medicine

“…[4][5][6][7] AI has been predicted to impact emergency medicine in the near future, with possible applications including patient outcome prediction, early identification of deterioration through vital sign monitoring and clinical image analysis. [8][9][10] However, the implementation of new technologies in the health care setting is complex, with acceptance of new technologies differing between individuals and workplaces. 11,12 The majority of previous research assessing attitudes towards AI in healthcare has been survey-based, with a focus on radiologists and medical students.…”