Evaluation of a digital system to predict unplanned admissions to the intensive care unit: A mixed-methods approach

Abstract

Background We have developed the Hospital Alerting Via Electronic Noticeboard (HAVEN) which aims to identify hospitalised patients most at risk of reversible deterioration. HAVEN combines patients’ vital-sign measurements with laboratory results, demographics and comorbidities using a machine learnt algorithm. Objectives The aim of this study was to identify variables or concepts that could improve HAVEN predictive performance. Methods This was an embedded, mixed methods study. Eligible patients with the five highest HAVEN scores in the hospital (i.e., ‘HAVEN Top 5′) had their medical identification details recorded. We conducted a structured medical note review on these patients 48 hours post their identifiers being recorded. Methods of constant comparison were used during data collection and to analyse patient data. Results The 129 patients not admitted to ICU then underwent constant comparison review, which produced three main groups. Group 1 were patients referred to specialist services (n = 37). Group 2 responded to ward-based treatment, (n = 38). Group 3 were frail and had documented treatment limitations (n = 47). Conclusions Digital-only validation methods code the cohort not admitted to ICU as ‘falsely positive’ in sensitivity analyses however this approach limits the evaluation of model performance. Our study suggested that coding for patients referred to other specialist teams, those with treatment limitations in place, along with those who are deteriorating but then respond to ward-based therapies, would give a more accurate measure of the value of the scores, especially in relation to cost-effectiveness of resource utilisation.

Protocol describing a systematic review and mixed methods consensus process to define the deteriorated ward patient

“… 2 The decision to transfer patients to the ICU is dependent on multiple factors, including personalised advance care directives, clinician opinion, local care escalation protocols such as early warning score (EWS) systems, and the availability of ICU resources. 3 Cardiac arrest and death are well defined and easily measured but are often a very late marker of deterioration. Additionally, cardiac arrest frequency is rare, which limits its use for derivation and validation processes, even in large patient data sets.…”

“…These include unplanned transfer from the general ward to the intensive care unit (ICU), cardiac arrest and death 2. The decision to transfer patients to the ICU is dependent on multiple factors, including personalised advance care directives, clinician opinion, local care escalation protocols such as early warning score (EWS) systems, and the availability of ICU resources 3. Cardiac arrest and death are well defined and easily measured but are often a very late marker of deterioration.…”