Machine Learning Algorithms to Identify Patients at Risk of Ventilator-Associated Pneumonia

Abstract

Rationale Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). However, classic clinical indicators of VAP demonstrate poor accuracy; relying solely on them to detect VAP may delay antimicrobial therapy. ​VAP diagnostics derived from machine learning methods that utilize electronic health record data have not yet been explored. The objective of this study was to compare the performance of a variety of machine learning models trained to predict VAP diagnosis. Methods A retrospective study examined data from 6,129 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of an International Classification of Diseases, 9th revision code for VAP. Models were trained to generate predictions 48 hours after initiation of mechanical ventilation, by definition the first possible time of VAP onset. Predictions therefore functioned as either onset alerts or prediction of future VAP. The following five types of models were trained using five-fold cross-validation: logistic regression, multilayer perceptron, random forest, support vector machine, and XGBoost gradient boosted decision tree. Each model was evaluated for its performance generating predictions using the prior 6, 12, 24, and 48 hours of patient data. Model performance was evaluated by the area under the receiver operating characteristic curve (AUROC) on a 10% hold-out test set. Feature importance was measured with Shapley values. Results The highest performing model was XGBoost using the prior 6 hours of patient data; this model achieved an AUROC of 0.828. XGBoost demonstrated the strongest performance at all prediction windows except the 48 hour window, for which logistic regression demonstrated the highest AUROC. The most important features for the best-performing model were the length of time on mechanical ventilation, treatment with antibiotics, sputum test frequency, and most recent Glasgow Coma Scale assessment. Conclusions VAP diagnostics derived from the application of supervised machine learning to electronic health record data are promising and should receive further validation alongside classic clinical indicators. Such tools have the potential to aid in the timely diagnosis of VAP.