Multicenter validation of a machine-learning algorithm for 48-h all-cause mortality prediction

Mohamadlou, Hamid; Panchavati, Saarang; Calvert, Jacob; Lynn-Palevsky, Anna; Le, Sidney; Allen, Angier; Pellegrini, Emily; Green‐Saxena, Abigail; Barton, Christopher; Fletcher, Grant; Shieh, Lisa; Stark, Philip B.; Chettipally, Uli K.; Shimabukuro, David; Feldman, Mitchell D.; Das, Ritankar

doi:10.1177/1460458219894494

Cited by 20 publications

(25 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found that the ML algorithm predicted the need for mechanical ventilation within 24 h among COVID-19 patients with high sensitivity and specificity. This work builds upon our prior work developing algorithms to predict patient outcomes including sepsis [ 15 ], acute kidney injury [ 16 ], mortality [ 17 ], and patient stability and decompensation [ 18 ]. While machine learning algorithms have been applied to retrospective COVID-19 patient data, no equivalent algorithms have yet been validated in a prospective setting, despite urgent need.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial

Burdick

Lam

Mataraso

et al. 2020

Computers in Biology and Medicine

Self Cite

136

View full text Add to dashboard Cite

Background Currently, physicians are limited in their ability to provide an accurate prognosis for COVID-19 positive patients. Existing scoring systems have been ineffective for identifying patient decompensation. Machine learning (ML) may offer an alternative strategy. A prospectively validated method to predict the need for ventilation in COVID-19 patients is essential to help triage patients, allocate resources, and prevent emergency intubations and their associated risks. Methods In a multicenter clinical trial, we evaluated the performance of a machine learning algorithm for prediction of invasive mechanical ventilation of COVID-19 patients within 24 h of an initial encounter. We enrolled patients with a COVID-19 diagnosis who were admitted to five United States health systems between March 24 and May 4, 2020. Results 197 patients were enrolled in the REspirAtory Decompensation and model for the triage of covid-19 patients: a prospective studY (READY) clinical trial. The algorithm had a higher diagnostic odds ratio (DOR, 12.58) for predicting ventilation than a comparator early warning system, the Modified Early Warning Score (MEWS). The algorithm also achieved significantly higher sensitivity (0.90) than MEWS, which achieved a sensitivity of 0.78, while maintaining a higher specificity (p < 0.05). Conclusions In the first clinical trial of a machine learning algorithm for ventilation needs among COVID-19 patients, the algorithm demonstrated accurate prediction of the need for mechanical ventilation within 24 h. This algorithm may help care teams effectively triage patients and allocate resources. Further, the algorithm is capable of accurately identifying 16% more patients than a widely used scoring system while minimizing false positive results.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This study builds upon existing evidence about the ability of algorithms to successfully provide clinical decision support [ [15] , [16] , [17] , [18] ]. However, there are several limitations to this study.…”

Section: Discussionmentioning

confidence: 99%

Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial

Burdick

Lam

Mataraso

et al. 2020

Computers in Biology and Medicine

Self Cite

136

View full text Add to dashboard Cite

show abstract

“…Mortality prediction tools aid in triage and resource allocation by providing advance warning of patient deterioration. Our prior work has validated machine-learning (ML) algorithms for their ability to predict mortality and patient stability in a variety of settings and on diverse patient populations [ [20] , [21] , [22] , [23] , [24] ].…”

Section: Introductionmentioning

confidence: 99%

Mortality prediction model for the triage of COVID-19, pneumonia, and mechanically ventilated ICU patients: A retrospective study

Ryan¹,

Lam²,

Mataraso³

et al. 2020

Annals of Medicine and Surgery

Self Cite

View full text Add to dashboard Cite

Rationale Prediction of patients at risk for mortality can help triage patients and assist in resource allocation. Objectives Develop and evaluate a machine learning-based algorithm which accurately predicts mortality in COVID-19, pneumonia, and mechanically ventilated patients. Methods Retrospective study of 53,001 total ICU patients, including 9166 patients with pneumonia and 25,895 mechanically ventilated patients, performed on the MIMIC dataset. An additional retrospective analysis was performed on a community hospital dataset containing 114 patients positive for SARS-COV-2 by PCR test. The outcome of interest was in-hospital patient mortality. Results When trained and tested on the MIMIC dataset, the XGBoost predictor obtained area under the receiver operating characteristic (AUROC) values of 0.82, 0.81, 0.77, and 0.75 for mortality prediction on mechanically ventilated patients at 12-, 24-, 48-, and 72- hour windows, respectively, and AUROCs of 0.87, 0.78, 0.77, and 0.734 for mortality prediction on pneumonia patients at 12-, 24-, 48-, and 72- hour windows, respectively. The predictor outperformed the qSOFA, MEWS and CURB-65 risk scores at all prediction windows. When tested on the community hospital dataset, the predictor obtained AUROCs of 0.91, 0.90, 0.86, and 0.87 for mortality prediction on COVID-19 patients at 12-, 24-, 48-, and 72- hour windows, respectively, outperforming the qSOFA, MEWS and CURB-65 risk scores at all prediction windows. Conclusions This machine learning-based algorithm is a useful predictive tool for anticipating patient mortality at clinically useful, and is capable of accurate mortality prediction for mechanically ventilated patients as well as those diagnosed with pneumonia and COVID-19.

show abstract

“…In 17 studies, the authors reported their model as more useful or superior to the EWS. [20][21][22][23][26][27][28]34,[36][37][38][39][40][41] Four studies reported real-time detection of deterioration before regular EWS, 20,26,42 and three studies reported positive effects on patient-related outcomes. 26,35 Four negative effects were noted on the controllability, validity, and potential limitations.…”

Section: Effects Facilitators and Barriersmentioning

confidence: 99%

Algorithms for Prediction of Clinical Deterioration on the General Wards: A Scoping Review

Peelen

Eddahchouri

Koeneman

et al. 2021

Journal of Hospital Medicine

View full text Add to dashboard Cite

OBJECTIVE: The primary objective of this scoping review was to identify and describe state-of-the-art models that use vital sign monitoring to predict clinical deterioration on the general ward. The secondary objective was to identify facilitators, barriers, and effects of implementing these models. DATA SOURCES: PubMed, Embase, and CINAHL databases until November 2020. STUDY SELECTION: We selected studies that compared vital signs–based automated real-time predictive algorithms to current track-and-trace protocols in regard to the outcome of clinical deterioration in a general ward population. DATA EXTRACTION: Study characteristics, predictive characteristics and barriers, facilitators, and effects. RESULTS: We identified 1,741 publications, 21 of which were included in our review. Two of the these were clinical trials, 2 were prospective observational studies, and the remaining 17 were retrospective studies. All of the studies focused on hospitalized adult patients. The reported area under the receiver operating characteristic curves ranged between 0.65 and 0.95 for the outcome of clinical deterioration. Positive predictive value and sensitivity ranged between 0.223 and 0.773 and 7.2% to 84.0%, respectively. Input variables differed widely, and predicted endpoints were inconsistently defined. We identified 57 facilitators and 48 barriers to the implementation of these models. We found 68 reported effects, 57 of which were positive. CONCLUSION: Predictive algorithms can detect clinical deterioration on the general ward earlier and more accurately than conventional protocols, which in one recent study led to lower mortality. Consensus is needed on input variables, predictive time horizons, and definitions of endpoints to better facilitate comparative research.

show abstract

Multicenter validation of a machine-learning algorithm for 48-h all-cause mortality prediction

Cited by 20 publications

References 40 publications

Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial

Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial

Mortality prediction model for the triage of COVID-19, pneumonia, and mechanically ventilated ICU patients: A retrospective study

Algorithms for Prediction of Clinical Deterioration on the General Wards: A Scoping Review

Contact Info

Product

Resources

About