Predicting and Communicating Risk of Clinical Deterioration: An Observational Cohort Study of Internal Medicine Residents

Abstract

BACKGROUND: Despite its importance, little is known about internal medicine (IM) residents' ability to assess and communicate a patient's overnight risk during the resident-to-resident handoff. OBJECTIVE: To evaluate IM residents' ability to identify patients at risk for clinical deterioration using the Patient Acuity Rating (PAR) tool (scored on a 1-7 symmetric scale; 1="Extremely unlikely", 7="Extremely likely"), and to measure how well IM residents conveyed a patient's potential for clinical deterioration during day-to-night handoff. DESIGN AND PARTICIPANTS: Observational cohort study of 46 postgraduate year 1 (PGY-1) and 32 postgraduate year 3 (PGY-3) internal medicine residents rotating on one of four general medicine services from October 2013 through January 2014. MAIN MEASURES: Primary outcomes were (1) level of agreement between resident handoff giver and receiver regarding patients' clinical risk and (2) accuracy of resident-assigned PAR score in predicting a patient's risk of clinical deterioration over the subsequent 24 hours. KEY RESULTS: Analysis of PGY-1 giver-receiver handoff agreement revealed an intraclass correlation coefficient (ICC) (95 % CI) of 0.51 (0.45-0.56), while PGY-3 giverreceiver agreement yielded an ICC (95 % CI) of 0.42 (0.36-0.47). Based on 865 ratings of 378 patients, PGY-1 handoff giver PAR scores of 5 and 6+ were significantly associated with increased odds of clinical deterioration within 24 hours (aOR=6.5 and 12.4; P=0.03 and 0.005, respectively). For the 1,170 PAR ratings of 438 patients assigned by PGY-3 handoff givers, PAR scores of 4, 5, and 6+ were significantly associated with increased odds of an event within 24 hours (aORs=6.0, 9.6, and 18.1; P=0.03, 0.01, and 0.0008, respectively). CONCLUSIONS: The PAR is a useful tool to quantify IM residents' judgment of patient stability, and may be particularly valuable during resident handoff, given that the level of agreement between giver and receiver regarding patient risk is only fair.

Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study

“…The rate of clinical deterioration (1.9%) observed in our study was similar to that in previous studies in similar clinical environments 18 19. The AUROC for intern physicians using the probability scale in our study (0.70) was similar to that observed with physicians on general internal medicine teaching services using the single-question, 7-point Likert scale patient acuity rating 18 19.…”

“…Physicians’ ability to predict, on admission to the ICU, which patients will ultimately die has been well studied, with a pooled AUROC of 0.85 across eight studies 18. Studies in general medical wards have found that physicians have an AUROC of 0.69–0.84 for predicting clinical deterioration at 24 hours 19 20. Prospectively comparing an EWS to physician prediction, the SUPPORT prognostic model demonstrated equal predictive ability for 180-day mortality compared with ICU physicians and AUROC 0.78 for both 21.…”

Incorporating Patient Acuity Rating Score Into Patient Handoffs and the Correlation With Rapid Responses and Unexpected ICU Transfers

“…8 Recently, the use of a Physician Acuity Rating (PAR) system was validated and found to be a predictor of severity of illness and the likelihood of transfer to a critical care unit. 7 The PAR scale is a symmetric scale used to score how likely a physician deems a patient is to 7,9,10 It has also been shown that this scale can assist house staff in communicating the severity of a patient's illness for the purpose of overnight cross-coverage. 9 A standardized method has not been endorsed for conveying information critical to patient care, 11 but many studies have favored best practice techniques 12,13 and communication mnemonics.…”

“…PAR scores could be a beneficial addition to attending physician handoffs as a means to increase vigilance for patients at risk for decompensating while cross-covering and for patient triage. 9,10 At the study institution, historical data showed that there were almost 400 RRT activations in 2012, with approximately half called within the first 24 hours of admission. Identifying patients at risk for decompensation would be useful for those physicians who are tasked with managing cross-coverage of almost 90 patients overnight.…”

“…7 The PAR scale is a symmetric scale used to score how likely a physician deems a patient is to decompensate: 1 = extremely unlikely , 7 = extremely likely . 7,9,10 It has also been shown that this scale can assist house staff in communicating the severity of a patient’s illness for the purpose of overnight cross-coverage. 9 A standardized method has not been endorsed for conveying information critical to patient care, 11 but many studies have favored best practice techniques 12,13 and communication mnemonics.…”