The criticality Index-mortality: A dynamic machine learning prediction algorithm for mortality prediction in children cared for in an ICU

Abstract

BackgroundThe Criticality Index-Mortality uses physiology, therapy, and intensity of care to compute mortality risk for pediatric ICU patients. If the frequency of mortality risk computations were increased to every 3 h with model performance that could improve the assessment of severity of illness, it could be utilized to monitor patients for significant mortality risk change.ObjectivesTo assess the performance of a dynamic method of updating mortality risk every 3 h using the Criticality Index-Mortality methodology and identify variables that are significant contributors to mortality risk predictions.PopulationThere were 8,399 pediatric ICU admissions with 312 (3.7%) deaths from January 1, 2018 to February 29, 2020. We randomly selected 75% of patients for training, 13% for validation, and 12% for testing.ModelA neural network was trained to predict hospital survival or death during or following an ICU admission. Variables included age, gender, laboratory tests, vital signs, medications categories, and mechanical ventilation variables. The neural network was calibrated to mortality risk using nonparametric logistic regression.ResultsDiscrimination assessed across all time periods found an AUROC of 0.851 (0.841–0.862) and an AUPRC was 0.443 (0.417–0.467). When assessed for performance every 3 h, the AUROCs had a minimum value of 0.778 (0.689–0.867) and a maximum value of 0.885 (0.841,0.862); the AUPRCs had a minimum value 0.148 (0.058–0.328) and a maximum value of 0.499 (0.229–0.769). The calibration plot had an intercept of 0.011, a slope of 0.956, and the R2 was 0.814. Comparison of observed vs. expected proportion of deaths revealed that 95.8% of the 543 risk intervals were not statistically significantly different. Construct validity assessed by death and survivor risk trajectories analyzed by mortality risk quartiles and 7 high and low risk diseases confirmed a priori clinical expectations about the trajectories of death and survivors.ConclusionsThe Criticality Index-Mortality computing mortality risk every 3 h for pediatric ICU patients has model performance that could enhance the clinical assessment of severity of illness. The overall Criticality Index-Mortality framework was effectively applied to develop an institutionally specific, and clinically relevant model for dynamic risk assessment of pediatric ICU patients.

Clinical Instability Is a Sign of Severity of Illness: A Cohort Study

“…This table is modified from Hong et al (8). Diagnoses were categorized by the International Classification of Diseases, 10th Edition (10). All diagnoses were categorized as system of dysfunction so there were greater than one/patient.…”

“…Details regarding the data and mortality risk modeling have been published (9, 10). The study was performed at Children’s National Hospital, an academic hospital with 323 beds inclusive of 48 bed PICU beds and 24 cardiac ICU beds.…”

“…Data cleaning and analysis used R software (Version 4.0.2) and Tidyverse (Version 1.3.2) (11, 12). Data included descriptive information, outcome, and independent variables used in the Criticality Index-Mortality (CI-M) (see Below ) (9, 10). Medication data were extracted from the medication administration record using start and end times and categorized using the Multum system (Cerner Corporation, Kansas City, MO) (13).…”

“…Medication data were extracted from the medication administration record using start and end times and categorized using the Multum system (Cerner Corporation, Kansas City, MO) (13). A comprehensive list of independent variables and details of medication classification have been previously published (9, 10, 14). Diagnoses were categorized by the International Classification of Diseases , 10th Edition (15).…”

“…The analysis was truncated at 11.4 days (273 hr) due to decrease in sample size secondary to discharges and deaths. Specifics of the modeling have been published (10). In brief, the final machine learning model has three hidden layers with 32 nodes each, bias and Rectified Linear Unit activation functions, L2 regularization with a parameter 0.02, and drop out nodes of 0.05.…”