Transferability and interpretability of the sepsis prediction models in the intensive care unit

Abstract

Background We aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples. Methods Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital. Results The datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 − 0.98 and 0.95 − 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 − 0.86 and 0.84 − 0.87 respectively on HDRJH, from 1 to 5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 − 0.94 on HDRJH and to 0.86 − 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable. Conclusions Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention. Trial registration: NCT05088850 (retrospectively registered).

A Time Series Driven Model for Early Sepsis Prediction Based on Transformer Module

“…Therefore, transfer learning may be a promising and viable strategy to maintain the effectiveness of machine learning models across multicenter deployments. Chen et al improved the performance of LightGBM and MLP models in predicting sepsis occurrence within 1-5 hours using Transformer, achieving favorable areas under the receiver characteristic curve (AUC) within the range of 0.96-0.98 [20]. In our study, the process of transfer learning effectively enhanced the performance of RNN and LSTM models in predicting sepsis occurrence in the e-ICU dataset.…”

A comprehensive ML-based Respiratory Monitoring System for Physiological Monitoring & Resource Planning in the ICU

“…Investigating and quantifying the underlying policy differences, which make transfer difficult, needs additional research. Model transferability is an emergent topic in robust machine learning for ICU settings, and recent works study it for sepsis 13,52,53 or mortality prediction 54 . Our results suggest that medication variables require special attention to enable transfer.…”

A time series driven model for early sepsis prediction based on transformer module

“…Therefore, transfer learning may be a promising and viable strategy to maintain the effectiveness of machine learning models across multi-center deployments. Chen et al improved the performance of LightGBM and MLP models in predicting sepsis occurrence within 1–5 h using Transformer, achieving favorable areas under the receiver operating characteristic curve (AUC) within the range of 0.96–0.98 [ 20 ]. In our study, the process of transfer learning effectively enhanced the performance of RNN and LSTM models in predicting sepsis occurrence in the e-ICU dataset.…”