Development and Validation of an Interpretable 3-day Intensive Care Unit Readmission Prediction Model Using Explainable Boosting Machines

Abstract

Intensive care unit readmissions are associated with mortality and bad outcomes. Machine learning could help to identify patients at risk to improve discharge decisions. However, many models are black boxes, so that dangerous properties might remain unnoticed. In this study, an inherently interpretable model for 3-day ICU readmission prediction was developed. We used a retrospective cohort of 15,589 ICU stays and 169 variables collected between 2006 and 2019. A team of doctors inspected the model, checked the plausibility of each component, and removed problematic parts. Qualitative feedback revealed several challenges for interpretable machine learning in healthcare. The resulting model used 67 features and showed an area under the precision-recall curve of 0.119±0.020 and an area under the receiver operating characteristic curve of 0.680±0.025. This is on par with state-of-the-art gradient boosting machines and outperforms the Simplified Acute Physiology Score II. External validation with the Medical Information Mart for Intensive Care database version IV confirmed our findings. Hence, a machine learning model for readmission prediction with a high level of human control is feasible without sacrificing performance.

Development and validation of an interpretable 3 day intensive care unit readmission prediction model using explainable boosting machines

“…We provided the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) checklist ( 28 ) in Supplementary material 1 . This work was preregistered online ( 29 ); however, it had two deviations: a readmission interval of 3 days instead of 7 days was considered to exclude fewer patients with insufficient follow-ups. Also, we only performed external validation for the final performance results, which we considered most relevant.…”

“…An overview of all steps conducted for this study can be found in Figure 1 . All code for preprocessing the data, training the models, and inspecting the final EBM model is publicly available ( 30 , 31 ).…”

“…The authors would like to thank Oliver Wenning for helping with the software development and Monica Agrawal for proofreading the article. This manuscript has been released as a preprint at medRxiv ( 58 ).…”

An Interpretable Two-Phase Modeling Approach for Lung Cancer Survivability Prediction

“…Cox R, Kaplan-Meier [13] SEER 2004-2016, # of variables is not disclosed, classification + regression for 3 categories: ≤6 months, 7-24 months, and ≥24 months ANN, RNN, CNN, RF, SVM, NB, GBM, LR [7] SEER 2010-2015, 12 variables, classification (1-, 3-, 5-year survival) XGB, LR, NB, DT, KNN, RF, SVM [14] SEER 2010-2015, 14 variables, classification (5-year survival) LR, NB, Gaussian K-base NB, [15] SEER 1973-2012, 114 variables, classification (0.5-, 1-, 5-year survival) RF, ANN Data Mining [16] SEER 2004-2009, 13 variables, classification + regression for 3 categories: ≤6 months, 7-24 months, and ≥24 months GBM, RF, GLM, EV Survival status prediction, length of survival estimation, and cancer patient clustering are primary topics found in the machine learning literature that utilizes the SEER dataset, where focus is placed on model accuracy. Moreover, common classification, clustering, and regression models employed within the second group of research include artificial neural networks (ANNs), support vector machines (SVMs), Naïve Bayes (NB), decision trees (DTs), random forest (RF), ensemble methods, K-means, and bidirectional data partitioning (BDP) [7,[13][14][15][16][17][18][19][20][21][22]. Apart from the great strides made in lung cancer prediction research, several challenges still exist:…”

“…Although most studies explore survival status classification [7,14,15,17,18,21,22] and survival length prediction [19] individually, a scheme that leverages both remains elusive [13,16]. • Data used in lung cancer survivability predictions suffer from the class imbalance problem, which produces algorithm bias in favor of the majority class.…”