Beyond Reporting Early Warning Score Sensitivity: The Temporal Relationship and Clinical Relevance of “True Positive” Alerts that Precede Critical Deterioration

Winter, Meredith C.; Kubis, S.; Bonafide, Christopher P.

doi:10.12788/jhm.3066

Cited by 15 publications

(11 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PEWS, like all point scores of its kind including the Pediatric Rothman Index [32], takes snapshots of clinical status at the time of nurse vital sign assessments. Much can happen clinically between these intermittent events-here, there is a role for continuous physiological monitoring as a means for detecting clinical deterioration.…”

Section: Discussionmentioning

confidence: 99%

Accuracy and Monitoring of Pediatric Early Warning Score (PEWS) Scores Prior to Emergent Pediatric Intensive Care Unit (ICU) Transfer: Retrospective Analysis (Preprint)

Kowalski¹,

Lee²,

Spaeder³

et al. 2020

Preprint

View full text Add to dashboard Cite

BACKGROUND Current approaches to early detection of clinical deterioration in children have relied on intermittent track-and-trigger warning scores such as the Pediatric Early Warning Score (PEWS) that rely on periodic assessment and vital sign entry. There are limited data on the utility of these scores prior to events of decompensation leading to pediatric intensive care unit (PICU) transfer. OBJECTIVE The purpose of our study was to determine the accuracy of recorded PEWS scores, assess clinical reasons for transfer, and describe the monitoring practices prior to PICU transfer involving acute decompensation. METHODS We conducted a retrospective cohort study of patients ≤21 years of age transferred emergently from the acute care pediatric floor to the PICU due to clinical deterioration over an 8-year period. Clinical charts were abstracted to (1) determine the clinical reason for transfer, (2) quantify the frequency of physiological monitoring prior to transfer, and (3) assess the timing and accuracy of the PEWS scores 24 hours prior to transfer. RESULTS During the 8-year period, 72 children and adolescents had an emergent PICU transfer due to clinical deterioration, most often due to acute respiratory distress. Only 35% (25/72) of the sample was on continuous telemetry or pulse oximetry monitoring prior to the transfer event, and 47% (34/72) had at least one incorrectly documented PEWS score in the 24 hours prior to the event, with a score underreporting the actual severity of illness. CONCLUSIONS This analysis provides support for the routine assessment of clinical deterioration and advocates for more research focused on the use and utility of continuous cardiorespiratory monitoring for patients at risk for emergent transfer.

show abstract

Section: Discussionmentioning

confidence: 99%

Accuracy and Monitoring of Pediatric Early Warning Score (PEWS) Scores Prior to Emergent Pediatric Intensive Care Unit (ICU) Transfer: Retrospective Analysis (Preprint)

Kowalski¹,

Lee²,

Spaeder³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Patient develops a fever Patient becomes more somnolent Hypotension develops of the percentage of cases that recognize the event prior to clinicians. 15…”

Section: Operationalizing Timing In Ews Training and Evaluationmentioning

confidence: 99%

“…By contrast, the sepsis detection models used in administrative databases rely on an encounter‐level PPV, which provides more limited information compared with real‐time EWSs 17 . When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians 15 …”

Section: Operationalizing Timing In Ews Training and Evaluationmentioning

confidence: 99%

“…17 When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians. 15…”

Section: Operationalizing Timing In Ews Training and Evaluationmentioning

confidence: 99%

“…On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies. 15…”

mentioning

confidence: 99%

See 2 more Smart Citations

Early Warning Systems: The Neglected Importance of Timing

Rolnick

Weissman

2019

J Hosp Med

View full text Add to dashboard Cite

A utomated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown. [1][2][3][4] We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development. 5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames. 9 We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section. CLINICAL TIME ZERO AND POSITIVE PREDICTIVE VALUEEWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. How-ever, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS's usefulness to promote interventions and generate increased alert fatigue. 10 However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero. 8,[11][12][13] All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used. 14 Further work is needed to gain additional insights into the measurement of time zero.Just as the sa...

show abstract

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit

Mehta

Congdon

Phillips

et al. 2023

Journal of Hospital Medicine

View full text Add to dashboard Cite

Background: Late recognition of in-hospital deterioration is a source of preventable harm. Emergency transfers (ET), when hospitalized patients require intensive care unit (ICU) interventions within 1 h of ICU transfer, are a proximal measure of late recognition associated with increased mortality and length of stay (LOS).Objective: To apply diagnostic process improvement frameworks to identify missed opportunities for improvement in diagnosis (MOID) in ETs and evaluate their association with outcomes.Design, Settings, and Participants: A single-center retrospective cohort study of ETs, January 2015 to June 2019. ET criteria include intubation, vasopressor initiation, or ≥60 mL/kg fluid resuscitation 1 h before to 1 h after ICU transfer. The primary exposure was the presence of MOID, determined using SaferDx. Cases were screened by an ICU and non-ICU physician. Final determinations were made by an interdisciplinary group. Diagnostic process improvement opportunities were identified.Main Outcome and Measures: Primary outcomes were in-hospital mortality and posttransfer LOS, analyzed by multivariable regression adjusting for age, service, deterioration category, and pretransfer LOS.Results: MOID was identified in 37 of 129 ETs (29%, 95% confidence interval [CI] 21%-37%). Cases with MOID differed in originating service, but not demographically. Recognizing the urgency of an identified condition was the most common diagnostic process opportunity. ET cases with MOID had higher odds of mortality (odds ratio 5.5; 95% CI 1.5−20.6; p = .01) and longer posttransfer LOS (rate ratio 1.7; 95% CI 1.1-2.6; p = .02). Conclusion: MOID are common in ETs and are associated with increased mortality risk and posttransfer LOS. Diagnostic improvement strategies should be leveraged to support earlier recognition of clinical deterioration.

show abstract

Beyond Reporting Early Warning Score Sensitivity: The Temporal Relationship and Clinical Relevance of “True Positive” Alerts that Precede Critical Deterioration

Abstract: Evidence that clinicians were aware of deterioration preceded pRI alerts in most CDEs that generated alerts in the preceding 24 hours.

Cited by 15 publications

References 17 publications

Accuracy and Monitoring of Pediatric Early Warning Score (PEWS) Scores Prior to Emergent Pediatric Intensive Care Unit (ICU) Transfer: Retrospective Analysis (Preprint)

Accuracy and Monitoring of Pediatric Early Warning Score (PEWS) Scores Prior to Emergent Pediatric Intensive Care Unit (ICU) Transfer: Retrospective Analysis (Preprint)

Early Warning Systems: The Neglected Importance of Timing

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit

Contact Info

Product

Resources

About