Prediction of In‐hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data–Driven, Machine Learning Approach

Abstract

Objectives: Predictive analytics in emergency care has mostly been limited to the use of clinical decision rules (CDRs) in the form of simple heuristics and scoring systems. In the development of CDRs, limitations in analytic methods and concerns with usability have generally constrained models to a preselected small set of variables judged to be clinically relevant and to rules that are easily calculated. Furthermore, CDRs frequently suffer from questions of generalizability, take years to develop, and lack the ability to be updated as new information becomes available. Newer analytic and machine learning techniques capable of harnessing the large number of variables that are already available through electronic health records (EHRs) may better predict patient outcomes and facilitate automation and deployment within clinical decision support systems. In this proof-of-concept study, a local, big datadriven, machine learning approach is compared to existing CDRs and traditional analytic methods using the prediction of sepsis in-hospital mortality as the use case.Methods: This was a retrospective study of adult ED visits admitted to the hospital meeting criteria for sepsis from October 2013 to October 2014. Sepsis was defined as meeting criteria for systemic inflammatory response syndrome with an infectious admitting diagnosis in the ED. ED visits were randomly partitioned into an 80%/20% split for training and validation. A random forest model (machine learning approach) was constructed using over 500 clinical variables from data available within the EHRs of four hospitals to predict in-hospital mortality. The machine learning prediction model was then compared to a classification and regression tree (CART) model, logistic regression model, and previously developed prediction tools on the validation data set using area under the receiver operating characteristic curve (AUC) and chi-square statistics.Results: There were 5,278 visits among 4,676 unique patients who met criteria for sepsis. Of the 4,222 patients in the training group, 210 (5.0%) died during hospitalization, and of the 1,056 patients in the validation group, 50 (4.7%) died during hospitalization. The AUCs with 95% confidence intervals (CIs) for the different models were as follows: random forest model, 0. Conclusions:In this proof-of-concept study, a local big data-driven, machine learning approach outperformed existing CDRs as well as traditional analytic techniques for predicting in-hospital mortality of ED patients with sepsis. Future research should prospectively evaluate the effectiveness of this approach and whether it translates into improved clinical outcomes for high-risk sepsis patients. The methods developed serve as an example of a new model for predictive analytics in emergency care that

Early sepsis detection in critical care patients using multiscale blood pressure and heart rate dynamics

“…Many predictions focus on death, an outcome for which there is little that can be directly modified in management [16–17]. Earlier recognition and diagnosis of adverse events such as sepsis has the potential to decrease morbidity and mortality through earlier initiation of treatment protocols.…”

Investigating the Impact of Different Suspicion of Infection Criteria on the Accuracy of Quick Sepsis-Related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores*

“…The proliferation of EHRs across the country has also spurred research using large datasets to develop models to identify high-risk patients with sepsis (7, 16, 17). Although ICD-9 codes can be used to identify patients for these studies, they lack a specific time component regarding the onset of sepsis.…”

“…The ability of the EHR to both collect and then calculate more complex scores makes simplifying models unnecessary (19). Several groups have already developed complex models for accurately identifying septic patients in the emergency department and on the wards (16, 17), with other studies showing improved processes of care due to early detection through EHR alerting (20–22). However, hospitals need to consider the relative improvements in accuracy of these complex models when compared to more standard scores, such as NEWS, in light of the challenges and costs that come with implementing machine learning models in practice.…”

An Algorithm Based on Deep Learning for Predicting In‐Hospital Cardiac Arrest

“…The details of SPTTS are shown in Table 3. In the previous studies, logistic regression and random forest were the most commonly used machine‐learning methods and showed better performance than traditional TTSs 32, 33. We used the area under the receiver operating characteristic curve (AUROC) and the area under the precision–recall curve (AUPRC) to measure the performance of the model.…”