Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach

Abstract

BackgroundSepsis is one of the leading causes of mortality in hospitalized patients. Despite this fact, a reliable means of predicting sepsis onset remains elusive. Early and accurate sepsis onset predictions could allow more aggressive and targeted therapy while maintaining antimicrobial stewardship. Existing detection methods suffer from low performance and often require time-consuming laboratory test results.ObjectiveTo study and validate a sepsis prediction method, InSight, for the new Sepsis-3 definitions in retrospective data, make predictions using a minimal set of variables from within the electronic health record data, compare the performance of this approach with existing scoring systems, and investigate the effects of data sparsity on InSight performance.MethodsWe apply InSight, a machine learning classification system that uses multivariable combinations of easily obtained patient data (vitals, peripheral capillary oxygen saturation, Glasgow Coma Score, and age), to predict sepsis using the retrospective Multiparameter Intelligent Monitoring in Intensive Care (MIMIC)-III dataset, restricted to intensive care unit (ICU) patients aged 15 years or more. Following the Sepsis-3 definitions of the sepsis syndrome, we compare the classification performance of InSight versus quick sequential organ failure assessment (qSOFA), modified early warning score (MEWS), systemic inflammatory response syndrome (SIRS), simplified acute physiology score (SAPS) II, and sequential organ failure assessment (SOFA) to determine whether or not patients will become septic at a fixed period of time before onset. We also test the robustness of the InSight system to random deletion of individual input observations.ResultsIn a test dataset with 11.3% sepsis prevalence, InSight produced superior classification performance compared with the alternative scores as measured by area under the receiver operating characteristic curves (AUROC) and area under precision-recall curves (APR). In detection of sepsis onset, InSight attains AUROC = 0.880 (SD 0.006) at onset time and APR = 0.595 (SD 0.016), both of which are superior to the performance attained by SIRS (AUROC: 0.609; APR: 0.160), qSOFA (AUROC: 0.772; APR: 0.277), and MEWS (AUROC: 0.803; APR: 0.327) computed concurrently, as well as SAPS II (AUROC: 0.700; APR: 0.225) and SOFA (AUROC: 0.725; APR: 0.284) computed at admission (P<.001 for all comparisons). Similar results are observed for 1-4 hours preceding sepsis onset. In experiments where approximately 60% of input data are deleted at random, InSight attains an AUROC of 0.781 (SD 0.013) and APR of 0.401 (SD 0.015) at sepsis onset time. Even with 60% of data missing, InSight remains superior to the corresponding SIRS scores (AUROC and APR, P<.001), qSOFA scores (P=.0095; P<.001) and superior to SOFA and SAPS II computed at admission (AUROC and APR, P<.001), where all of these comparison scores (except InSight) are computed without data deletion.ConclusionsDespite using little more than vitals, InSight is an effective tool for predic...

Early sepsis detection in critical care patients using multiscale blood pressure and heart rate dynamics

“…Our final model achieved an AUROC of 0.78 which is slightly higher than the existing works [18]. We anticipate that use of high resolution BP, HR and Resp dynamics, including waveform morphology and coupling features, will add further predictive power to our and other existing models and provide a more objective metric of early identification of sepsis and other types of decompensation in critically ill patients.…”

“…We note that the InSight algorithm [18] utilized a minimal set of commonly available EHR variables (including blood pressure, heart rate, respiration rate, temperature, SpO 2 , and GCS) to achieve a slightly lower AUC of 0.74 (± 0.01) for 4 hours ahead prediction of sepsis. However, the authors followed the sepsis III definition, they opted for predicting the time of two or more points change in the SOFA score (t sofa ), which is arguably easier to detect than t sepsis given the lab data.…”

“…In general, comparison of various algorithms within the literature has been made complicated due to scarcity of standardized publicly available ICU datasets, in addition to differences in definitions of sepsis and data elements collected across institutions. This may include utilization of acausal measurements, which occurs when the order-time of labs are used in prediction models [4, 18] rather than the time when laboratory data become available to the bedside clinicians. Therefore, validation and prospective studies are needed to shed light on the strengths and weaknesses of various algorithms.…”

Metabolomics in Sepsis and Its Impact on Public Health

“…Machine learning approaches have also been employed, complementary to clinical tools, in developing predictive models for sepsis; however, reliable diagnostic and predictive tools remain elusive [84]. …”

qSOFA—welcome to the sepsis alphabet soup

“…As Rothman contends that sepsis has two problems regarding identification at admission and predicting onset during hospitalization, we agree that sepsis screening models need to be developed to tailor to different settings (15). Because sepsis prevalence and mortality are different among populations of different settings, clinical criteria and prediction and prognostication models need to become more sophisticated dynamically and perhaps machine learning will help all of us in the near future (16). Even though a natural history of sepsis does exist, time zero on sepsis onset has yet to be defined.…”