Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19

Abstract

A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model’s performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature’s SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.

Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare

“…However, when the clinical environment is highly dynamic and patient populations are heterogeneous, a model that works well in one-time period or one hospital may fail in another. A recent example is the emergence of COVID-19 24 documented a performance drop in an ML algorithm for determining which patients were at high risk of hospital admission based on their emergency department (ED) presentation that relied on input variables like respiratory rate and arrival mode, which were significantly affected by the spread of COVID-19.…”

“…Given the complexity of ML algorithms, a number of papers have suggested monitoring ML explainability metrics, such as variable importance (VI) 18 , 24 . The idea is that these metrics provide a more interpretable representation of the data.…”

RapiD_AI: A framework for Rapidly Deployable AI for novel disease & pandemic preparedness

“…For XGBoost we tried to tune the number of estimators, (11,13,15,17,19), their depth (7,8,9,10,11,12,13) and learning rate (0.01, 0.1, 0.2, 0.3). Configuring neural networks is more difficult and sometimes computationally heavy, especially for the TabNet model.…”

“…Singh and colleagues [11] demonstrate the impact of distribution shifts across 179 US facilities on a sepsis prediction model. Duckworth and colleagues [12] show distribution shifts during the COVID-19 pandemic and evaluate their impact on a hospital admission prediction model.…”

Application of Machine Learning in Prediction of COVID-19 Diagnosis for Indonesian Healthcare Workers

“…It highlights timed-based drift in the data which influences the choice of threshold to be suboptimal. 40 Based on the AUC, AP, and Brier Score, random forest classifier was chosen for the Jakarta model due to well-calibrated predictions and high training and testing performance. SHAP analysis is then executed using random forest as the model.…”