Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods

Abstract

Background: COVID-19 has led to an unprecedented strain on health care facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here, we present the results of an analytical model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to identify patients at high risk for imminent intensive care unit transfer, respiratory failure, or death, with the intention to improve the prediction of deterioration due to COVID-19.Objective: This study aims to validate the PICTURE model's ability to predict unexpected deterioration in general ward and COVID-19 patients, and to compare its performance with the Epic Deterioration Index (EDI), an existing model that has recently been assessed for use in patients with COVID-19. Methods:The PICTURE model was trained and validated on a cohort of hospitalized non-COVID-19 patients using electronic health record data from 2014 to 2018. It was then applied to two holdout test sets: non-COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to EDI and NEWS scores for head-to-head comparison via area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve. We compared the models' ability to predict an adverse event (defined as intensive care unit transfer, mechanical ventilation use, or death). Shapley values were used to provide explanations for PICTURE predictions.

Detection of Hemodynamic Status Using an Analytic Based on an Electrocardiogram Lead Waveform

“…A multitude of clinical EWS have been developed and are variably used in less monitored inpatient settings to help identify or predict decompensation when less continuous data and vital signs are available. Examples include the MEWS, NEWS, eCART, and Predicting Intensive Care Transfers and Other Unforeseen Events and other scores that use intermittent vital signs and other data to assess risks for cardiac arrest, ICU transfer, or death ( 28 , 39 ). These EWS products are limited in their outputs since they are restricted to the intermittent nature of the data imputation required for scoring.…”

Early identification of patients admitted to hospital for covid-19 at risk of clinical deterioration: model development and multisite external validation study

“…Most notably, strong generalization performance (that is, how well a model will perform across different patient populations) is fundamental to realizing the potential benefits of risk models in clinical care. Yet generalization performance is often entirely overlooked when predictive models are developed and validated in healthcare 891011121314. For example, recent work found that only 5% of articles on predictive modeling in PubMed mention external validation in either the title or the abstract 9.…”

“…Yet generalization performance is often entirely overlooked when predictive models are developed and validated in healthcare. [8][9][10][11][12][13][14] For example, recent work found that only 5% of articles on predictive modeling in PubMed mention external validation in either the title or the abstract. 9 This is partly because most approaches to external validation require data sharing agreements.…”

Independent predictors of in-hospital mortality and the need for intensive care in hospitalized non-critical COVID-19 patients: a prospective cohort study

“…But a significant focus on inpatient management should be on ensuring a feasible, sustainable, and reliable risk categorization that can be adjusted to various non-critical cases. Recent studies have reported some nomograms and scoring systems; however, they did not satisfy the need for practical and reliable tools [12][13][14][15][16].…”