Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). Existing methods for identifying VAP display low accuracy, and their use may delay antimicrobial therapy. VAP diagnostics derived from machine learning (ML) methods that utilize electronic health record (EHR) data have not yet been explored. The objective of this study is to compare the performance of a variety of ML models trained to predict whether VAP will be diagnosed during the patient stay. A retrospective study examined data from 6126 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of a diagnostic code for VAP. Five different ML models were trained to predict VAP 48 hours after initiation of mechanical ventilation. Model performance was evaluated with regard to the area under the receiver operating characteristic (AUROC) curve on a 20% hold-out test set. Feature importance was measured in terms of Shapley values. The highest performing model achieved an AUROC value of 0.854. The most important features for the best-performing model were the length of time on mechanical ventilation, the presence of antibiotics, sputum test frequency, and the most recent Glasgow Coma Scale assessment. Supervised ML using patient EHR data is promising for VAP diagnosis and warrants further validation. This tool has the potential to aid the timely diagnosis of VAP.

Rationale Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). However, classic clinical indicators of VAP demonstrate poor accuracy; relying solely on them to detect VAP may delay antimicrobial therapy. ​VAP diagnostics derived from machine learning methods that utilize electronic health record data have not yet been explored. The objective of this study was to compare the performance of a variety of machine learning models trained to predict VAP diagnosis. Methods A retrospective study examined data from 6,129 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of an International Classification of Diseases, 9th revision code for VAP. Models were trained to generate predictions 48 hours after initiation of mechanical ventilation, by definition the first possible time of VAP onset. Predictions therefore functioned as either onset alerts or prediction of future VAP. The following five types of models were trained using five-fold cross-validation: logistic regression, multilayer perceptron, random forest, support vector machine, and XGBoost gradient boosted decision tree. Each model was evaluated for its performance generating predictions using the prior 6, 12, 24, and 48 hours of patient data. Model performance was evaluated by the area under the receiver operating characteristic curve (AUROC) on a 10% hold-out test set. Feature importance was measured with Shapley values. Results The highest performing model was XGBoost using the prior 6 hours of patient data; this model achieved an AUROC of 0.828. XGBoost demonstrated the strongest performance at all prediction windows except the 48 hour window, for which logistic regression demonstrated the highest AUROC. The most important features for the best-performing model were the length of time on mechanical ventilation, treatment with antibiotics, sputum test frequency, and most recent Glasgow Coma Scale assessment. Conclusions VAP diagnostics derived from the application of supervised machine learning to electronic health record data are promising and should receive further validation alongside classic clinical indicators. Such tools have the potential to aid in the timely diagnosis of VAP.

Objective Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). Existing methods for identifying VAP display low accuracy, and their use may delay antimicrobial therapy. ​VAP diagnostics derived from machine learning methods that utilize electronic health record data have not yet been explored. The objective of this study is to compare the performance of a variety of machine learning models trained to predict whether VAP will be diagnosed during the patient stay.Methods A retrospective study examined data from 6,129 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of a diagnostic code for VAP. Five different machine learning models were trained to predict VAP 48 hours after initiation of mechanical ventilation. Model performance was evaluated with regard to area under the receiver operating characteristic curve (AUROC) on a 10% hold-out test set. Feature importance was measured in terms of Shapley values.Results The highest performing model achieved an AUROC value of 0.827. The most important features for the best-performing model were the length of time on mechanical ventilation, presence of antibiotics, sputum test frequency, and most recent Glasgow Coma Scale assessment.Discussion Supervised machine learning using patient electronic health record data is promising for VAP diagnosis and warrants further validation. Conclusion This tool has the potential to aid the timely diagnosis of VAP.