Machine Learning for Real-Time Aggregated Prediction of Hospital Admission for Emergency Patients

Abstract

Machine learning for hospital operations is under-studied. We present a prediction pipeline that uses live electronic health-records for patients in a UK teaching hospital emergency department (ED) to generate short-term, probabilistic forecasts of emergency admissions. A set of XGBoost classifiers applied to 109,465 ED visits yielded AUROCs from 0.82 to 0.90 depending on elapsed visit-time at the point of prediction. Patient-level probabilities of admission were aggregated to forecast the number of admissions among current ED patients and, incorporating patients yet to arrive, total emergency admissions within specified time-windows. The pipeline gave a mean absolute error (MAE) of 4.0 admissions (mean percentage error of 17%) versus 6.5 (32%) for a benchmark metric. Models developed with 104,504 later visits during the Covid-19 pandemic gave AUROCs of 0.68-0.90 and MAE of 4.2 (30%) versus a 4.9 (33%) benchmark. We discuss how we surmounted challenges of designing and implementing models for real-time use, including temporal framing, data preparation, and changing operational conditions.

An external validation study of the Score for Emergency Risk Prediction (SERP), an interpretable machine learning-based triage score for the emergency department

“…Brajer et al suggested a machine-learning model fact sheet reporting for end-users 18 . Visualization-based efforts such as population, patient, and temporal level feature importance, or nomograms, could be adopted [19][20][21][22]…”

An external validation study of the Score for Emergency Risk Prediction (SERP), an interpretable machine learning-based triage score for the emergency department

“…Brajer et al suggested a machine-learning model fact sheet reporting for end-users 18 . Visualization-based efforts such as population, patient, and temporal level feature importance, or nomograms, could be adopted [19][20][21][22] . Like the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) or The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting machine learning results 23,24 , there should be a guideline for the standardization of user interfaces (UIs) and a format for clinical decision support for end-users, including clinicians and patients.…”

Performance Evaluation of Artificial Intelligence Methods Predicting Annual Number of Patients in Hospitals

“…인공지능기법을 보건의료에 적용한 연구에서는 응급실이나 중환자실의 환자 [9,10], 재원일수 [11,12], 재입원 [13,14], 진료비 [2] 등을 예측한 사례가 있다. 환자 수 예측에는 병원의 외래환자 수 예측을 위한 시계열 데이터처리 딥러닝 시스템 [10], 기계 적 학습방법을 이용한 응급실 환자 수 [9]가 있으며, 병원의 입원 재원기간 예측에 관한 연구들도 다수 존재한다 [15,16].…”