Background
Risk evaluation for device-associated infection and 30-day outcomes following the accomplishment of invasive device procedures is essential to patients admitted to the intensive care unit (ICU). We aimed to construct and validate machine learning (ML) models to predict the risk of device-associated infection and 30-day outcomes after invasive device procedures in ICU patients.
Methods
We included 8574 patients with ICU admission who accepted invasive device procedures from the Medical Information Mart for Intensive Care (MIMIC)-IV version 2.2 database. Enrolled patients were divided into development and test cohorts according to a proportion of 7:3. ML models were created based on the training dataset (n = 6001). We applied seven ML models for device-associated infection, including random forest (RF), logistic regression (LR), support vector machine (SVM), extreme gradient boosting (XGBoost), Gaussian naive Bayesian (GNB), decision tree (DT), and recurrent neural networks with long short-term memory (LSTM) algorithm. Five models being used for the 30-day survival outcome, including Cox regression, extra survival trees (EST), survival tree (ST), gradient boosting survival tree (GBST), and deep learning survival neural network (DeepSurv). The primary evaluated approaches to model performance were the receiver operating characteristic (ROC) curve for device-associated infection prediction and the survival model's concordance index (C-index). All models were internally validated in a test cohort (n = 2573).
Results
During the observation period of 30 days after invasive device procedures, 491 patients developed device-associated infections, and 1329 died. The XGBoost model presented the best-discriminated performance, with the test dataset's highest area under the curve (AUC) of 0.787 (0.787, 0.788), areas under the precision-recall curve (AUPRC) of 0.172 (0.172, 0.172), and the lowest Brier score (BS) of 0.146 (0.145, 0.146). The GBST model revealed the best ability to predict 30-day outcome survival, manifesting the highest C-index of 0.730 (0.728, 0.733) and time-dependent AUC of 0.744 (0.741, 0.748) in the validated cohort. The XGBoost and GBST have been available in the web application. These two models can generate an individual predictive risk of device-associated infection and 30-day survival outcomes for patients with ICU admission experiencing invasive device procedures.
Conclusions
We developed and internally evaluated XGBoost and GBST models with excellent prediction ability for the risk of device-associated infection and 30-day survival outcomes after invasive device procedures in patients hospitaled to ICU. The predictive result of these two models can help clinicians identify higher-risk patients with adverse events and conduct prevention methods.