Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study

Arnold, Jonathan; Davis, Alexander; Fischhoff, Baruch; Yecies, Emmanuelle; Grace, Jon; Klobuka, Andrew; Hanmer, Janel

doi:10.1136/bmjopen-2019-032187

Cited by 21 publications

(28 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One study reported both the retrospective development of a prediction model and a prospective observational study of the developed model [ 60 ]. The remaining studies included one randomized controlled trial [ 61 ], 4 before-after (implementation) studies [ 62 - 65 ], and 3 prospective observational studies [ 66 - 68 ].…”

Section: Resultsmentioning

confidence: 99%

“…Single and multiparameter scores were derived from a set of vital sign threshold derangements ( Tables 3 and 4 ). The statistical and machine learning methods included logistic regression, survival models, Cox regression, Gaussian process regression, Markov models, decision trees (random forest and gradient boosted trees), K-nearest neighbor, support vector machine, and neural networks ( Tables 1 and 2 , and Multimedia Appendix 1 [ 23 , 24 , 27 , 28 , 36 , 38 , 42 , 43 , 45 - 47 , 49 , 51 , 53 , 55 , 58 , 60 , 68 ]). Most of these models attempted to account for changes in physiologic measures over time using novel model frameworks, for example, taking a sliding time window looking forward or backward in time to predict outcomes.…”

Section: Resultsmentioning

confidence: 99%

“…All patient deterioration prediction tools used vital sign measures, most commonly blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, and a level of consciousness measure ( Multimedia Appendix 2 , Tables S2 and S3 [ 23 - 39 , 41 - 45 , 47 - 61 , 63 - 65 , 67 , 68 ]). In addition, most of the models included basic patient demographic data, such as age and gender, as well as administrative measures, such as admission status, time since admission, length of stay, and patient location.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

Mann¹,

Good²,

Fatehi³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Early warning tools identify patients at risk of deterioration in hospitals. Electronic medical records in hospitals offer real-time data and the opportunity to automate early warning tools and provide real-time, dynamic risk estimates. Objective This review describes published studies on the development, validation, and implementation of tools for predicting patient deterioration in general wards in hospitals. Methods An electronic database search of peer reviewed journal papers from 2008-2020 identified studies reporting the use of tools and algorithms for predicting patient deterioration, defined by unplanned transfer to the intensive care unit, cardiac arrest, or death. Studies conducted solely in intensive care units, emergency departments, or single diagnosis patient groups were excluded. Results A total of 46 publications were eligible for inclusion. These publications were heterogeneous in design, setting, and outcome measures. Most studies were retrospective studies using cohort data to develop, validate, or statistically evaluate prediction tools. The tools consisted of early warning, screening, or scoring systems based on physiologic data, as well as more complex algorithms developed to better represent real-time data, deal with complexities of longitudinal data, and warn of deterioration risk earlier. Only a few studies detailed the results of the implementation of deterioration warning tools. Conclusions Despite relative progress in the development of algorithms to predict patient deterioration, the literature has not shown that the deployment or implementation of such algorithms is reproducibly associated with improvements in patient outcomes. Further work is needed to realize the potential of automated predictions and update dynamic risk estimates as part of an operational early warning system for inpatient deterioration.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

Mann¹,

Good²,

Fatehi³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Identifying patients at risk for clinical deterioration with an automated system is vital for prioritising resources in a hospital setting. The automated early warning system has similar capabilities with medical predictions (Arnold et al, 2019). Thus, the prevention of harm with AI is associated with the adoption of a correct technology system code.…”

Section: Structure Issue: System With Artificial Intelligencementioning

confidence: 99%

“…AEWS equipment is available here and is efficient in the timely detection of a deteriorating patient. An early warning system works well with trained healthcare providers who have predictive qualities (Arnold et al, 2019). AEWSs increase communication and documentation, compared to manual early warning scores (Robb & Seddon, 2010).…”

Section: Structure Needs a System For Prediction And Prevention Of Harmmentioning

confidence: 99%

Patient Safety Culture in Perspective of Millennials: Case Study in a Malaysian Private Hospital

Singh¹,

Nasruddin

2020

IJIM

View full text Add to dashboard Cite

This case study investigated the nature of patient safety culture in a Malaysian private hospital. This was accomplished by studying the general perception of patient safety with a survey questionnaire and insights from open comments. The Donabedian framework of quality was used to further analyse the nature of patient safety culture from the perspective of millennials. Findings from the interviews were triangulated with data from observations and document reviews, which were interpreted based on the High Reliability Organisation (HRO) principles. They revealed that the structures and concerns in the practice environment regard the artificial intelligence (AI) system, human capital management, and manpower, while processes such as communication, teamwork and collaboration issues, and training and improvements are vital. Patient safety culture outcomes which depend on structures and processes indicated the need for improvements in the frequency of events reported. The main implication of this study is that to raise standards of care to those in an HRO and in a complex care environment, a paradigm shift from current reactive standards towards a more proactive safety system, based on flexibility and stability, would be necessary. This requires a strong case organisation to note appropriate structures and processes affecting eventual outcomes.

show abstract

Advances in Appropriate Postoperative Triage and the Role of Real-time Machine-Learning Models: The Goldilocks Dilemma

Wren

2021

JAMA Netw Open

View full text Add to dashboard Cite

It is critical in health care to neither overuse nor underuse resources. One of the key decisions in surgical treatment is to determine whether a patient can safely undergo an ambulatory procedure or requires hospital admission and observation. Patients admitted after a procedure undergo further stratification by admission status: to observation, inpatient ward, high-dependency unit, or intensive care unit (ICU). It is essential to accurately determine the optimal location for a patient after surgical treatment, balancing resource allocation with perioperative risks. A dynamic tension exists between overtriage to higher levels of care when not necessary, given that this is costly and is not associated with meaningful changes in perioperative outcomes, and conversely undertriage, which can also be associated with increased costs but may additionally be associated with worse perioperative outcomes. The Goldilocks admission dilemma of too hot, too cold, or just right has not yet been optimized. In this longitudinal, cross-sectional study, Loftus and associates 1 developed a machinelearning model to identify undertriage to hospital wards among patients after surgical procedures.They defined patients who were undertriaged as those who were sent to the floor but found to be in the top quartile of hospital mortality risk and who subsequently required a prolonged ICU stay of more than 48 hours. Out of more than 12 000 postoperative ward admissions during the 6-year study period, 10.6% of admissions were identified as undertriaged. By using this model, the authors found that, compared with risk-matched ICU admissions, undertriaged postoperative admissions had an increased risk of mortality and morbidity, namely unplanned intubation, acute kidney injury, and longer hospital length of stay.Existing models for postoperative triage have been primarily based on high-risk surgical procedures, as defined by estimated mortality rates of 5% or more, 2 or guided by basic tools, such as the Modified Early Warning Score (MEWS), calculated in the preoperative and postoperative setting. 3 The study by Loftus et al 1 is unique in that it takes machine-learning algorithms from preoperative and intraoperative data to estimate patients at increased risk of postoperative complications. In developing this model, the authors found that primary procedure, scheduled postoperative location, intraoperative minimum alveolar anesthetic concentration measurements, and duration of inhalation anesthetic were among the most important features of estimators associated with mortality and prolonged ICU stay.The authors point out that compared with appropriately triaged postoperative ward admissions and risk-matched control ICU admissions groups, the undertriaged admissions group was older, had an increased proportion of admitted patients identifying as Black or African American, and had increased area deprivation indices, suggesting increased socioeconomic disadvantage. The undertriaged admissions group also had increased Charlson Comorbidity Index scores...

show abstract

Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study

Cited by 21 publications

References 26 publications

Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

Patient Safety Culture in Perspective of Millennials: Case Study in a Malaysian Private Hospital

Advances in Appropriate Postoperative Triage and the Role of Real-time Machine-Learning Models: The Goldilocks Dilemma

Contact Info

Product

Resources

About