External validation of a prehospital risk score for critical illness

Abstract

BackgroundIdentification of critically ill patients during prehospital care could facilitate early treatment and aid in the regionalization of critical care. Tools to consistently identify those in the field with or at higher risk of developing critical illness do not exist. We sought to validate a prehospital critical illness risk score that uses objective clinical variables in a contemporary cohort of geographically and temporally distinct prehospital encounters.MethodsWe linked prehospital encounters at 21 emergency medical services (EMS) agencies to inpatient electronic health records at nine hospitals in southwestern Pennsylvania from 2010 to 2012. The primary outcome was critical illness during hospitalization, defined as an intensive care unit stay with delivery of organ support (mechanical ventilation or vasopressor use). We calculated the prehospital risk score using demographics and first vital signs from eligible EMS encounters, and we tested the association between score variables and critical illness using multivariable logistic regression. Discrimination was assessed using the AUROC curve, and calibration was determined by plotting observed versus expected events across score values. Operating characteristics were calculated at score thresholds.ResultsAmong 42,550 nontrauma, non-cardiac arrest adult EMS patients, 1926 (4.5 %) developed critical illness during hospitalization. We observed moderate discrimination of the prehospital critical illness risk score (AUROC 0.73, 95 % CI 0.72–0.74) and adequate calibration based on observed versus expected plots. At a score threshold of 2, sensitivity was 0.63 (95 % CI 0.61–0.75), specificity was 0.73 (95 % CI 0.72–0.73), negative predictive value was 0.98 (95 % CI 0.98–0.98), and positive predictive value was 0.10 (95 % CI 0.09–0.10). The risk score performance was greater with alternative definitions of critical illness, including in-hospital mortality (AUROC 0.77, 95 % CI 0.7 –0.78).ConclusionsIn an external validation cohort, a prehospital risk score using objective clinical data had moderate discrimination for critical illness during hospitalization.Electronic supplementary materialThe online version of this article (doi:10.1186/s13054-016-1408-0) contains supplementary material, which is available to authorized users.

A pre-hospital risk score predicts critical illness in non-trauma patients transported by ambulance to a Dutch tertiary referral hospital

“…In the current study, there was a relatively high frequency of critically ill patients of 12%. In previous studies this was 4.5% [8,10]. These studies included not only multiple referral centers of expertise, but also non-referral hospitals, possibly treating patients who were less ill.…”

“…This risk stratification tool has been developed in a United States population including non-trauma and non-cardiac arrest patients who were transported to the Emergency Departments of both non-referral hospitals, as well as regional referral centers. Kievlan, et al have validated the model for its use in a similar population in the United States [ 10 ]. This validation study showed an Area Under Receiver Operating Characteristics (AUROC) of 0.73.…”

A Multicenter Observational Prospective Cohort Study of Association of the Prehospital National Early Warning Score 2 and Hospital Triage with Early Mortality

“…We can find multiple studies on the Early Warning Score in the literature, but we opted for the NEWS2 because it is currently the most used in the PhEMS and has a high bibliographical consistency [8, 30].…”

Accuracy of National Early Warning Score 2 (NEWS2) in Prehospital Triage on In-Hospital Early Mortality: A Multi-Center Observational Prospective Cohort Study

“…These scales are composed of simple parameter systems, multiple parameters, or aggregated weighting systems, the latter being the most reliable. 22,30,31 For this study, four triage methods have been selected that can be performed (at least in a theoretical framework) in the context of MCIs at the prehospital level: SI, 32,33 GAP Score, 34,35 RTS, 36,37 and the NEWS2. 20,38 Table 1 shows the parameters that are evaluated in each scale.…”