Artificial intelligence predicts delirium following cardiac surgery: A case study

Fliegenschmidt, Janis; Hulde, Nikolai; Preising, Maria G.; Ruggeri, Silvia; Szymanowski, Ralph; Meesseman, Laurent; Sun, Hong; Dossow, Vera von

doi:10.1016/j.jclinane.2021.110473

Cited by 13 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, the preliminary evaluation made by the domain experts assures the correctness and effectiveness of the predictions. A case study has been conducted to evaluate the performance of the delirium prediction models installed in hospital H [ 21 ]. Predictions made by the delirium risk prediction model following cardiac surgery were evaluated in the study.…”

Section: Resultsmentioning

confidence: 99%

Machine Learning–Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance

Sun¹,

Depraetere²,

Meesseman³

et al. 2022

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Background Machine learning algorithms are currently used in a wide array of clinical domains to produce models that can predict clinical risk events. Most models are developed and evaluated with retrospective data, very few are evaluated in a clinical workflow, and even fewer report performances in different hospitals. In this study, we provide detailed evaluations of clinical risk prediction models in live clinical workflows for three different use cases in three different hospitals. Objective The main objective of this study was to evaluate clinical risk prediction models in live clinical workflows and compare their performance in these setting with their performance when using retrospective data. We also aimed at generalizing the results by applying our investigation to three different use cases in three different hospitals. Methods We trained clinical risk prediction models for three use cases (ie, delirium, sepsis, and acute kidney injury) in three different hospitals with retrospective data. We used machine learning and, specifically, deep learning to train models that were based on the Transformer model. The models were trained using a calibration tool that is common for all hospitals and use cases. The models had a common design but were calibrated using each hospital’s specific data. The models were deployed in these three hospitals and used in daily clinical practice. The predictions made by these models were logged and correlated with the diagnosis at discharge. We compared their performance with evaluations on retrospective data and conducted cross-hospital evaluations. Results The performance of the prediction models with data from live clinical workflows was similar to the performance with retrospective data. The average value of the area under the receiver operating characteristic curve (AUROC) decreased slightly by 0.6 percentage points (from 94.8% to 94.2% at discharge). The cross-hospital evaluations exhibited severely reduced performance: the average AUROC decreased by 8 percentage points (from 94.2% to 86.3% at discharge), which indicates the importance of model calibration with data from the deployment hospital. Conclusions Calibrating the prediction model with data from different deployment hospitals led to good performance in live settings. The performance degradation in the cross-hospital evaluation identified limitations in developing a generic model for different hospitals. Designing a generic process for model development to generate specialized prediction models for each hospital guarantees model performance in different hospitals.

show abstract

Section: Resultsmentioning

confidence: 99%

Machine Learning–Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance

Sun¹,

Depraetere²,

Meesseman³

et al. 2022

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sun and colleagues45 demonstrated their deep learning model performed equally well in training and prospective validation studies 29. In a subsequent case study, the authors demonstrated an instance where their application correctly predicted postoperative delirium in a patient with a negative preoperative CAM-ICU, demonstrating its clinical utility in a surgical ward 55. In addition, they found ML applications could be particularly useful for the early detection of delirium in wards where delirium screening is often not performed and delirium is underdiagnosed 1…”

Section: Discussionmentioning

confidence: 96%

Navigating the machine learning pipeline: a scoping review of inpatient delirium prediction models

Strating

Hanjani

Tornvall

et al. 2023

BMJ Health Care Inform

View full text Add to dashboard Cite

ObjectivesEarly identification of inpatients at risk of developing delirium and implementing preventive measures could avoid up to 40% of delirium cases. Machine learning (ML)-based prediction models may enable risk stratification and targeted intervention, but establishing their current evolutionary status requires a scoping review of recent literature.MethodsWe searched ten databases up to June 2022 for studies of ML-based delirium prediction models. Eligible criteria comprised: use of at least one ML prediction method in an adult hospital inpatient population; published in English; reporting at least one performance measure (area under receiver-operator curve (AUROC), sensitivity, specificity, positive or negative predictive value). Included models were categorised by their stage of maturation and assessed for performance, utility and user acceptance in clinical practice.ResultsAmong 921 screened studies, 39 met eligibility criteria. In-silico performance was consistently high (median AUROC: 0.85); however, only six articles (15.4%) reported external validation, revealing degraded performance (median AUROC: 0.75). Three studies (7.7%) of models deployed within clinical workflows reported high accuracy (median AUROC: 0.92) and high user acceptance.DiscussionML models have potential to identify inpatients at risk of developing delirium before symptom onset. However, few models were externally validated and even fewer underwent prospective evaluation in clinical settings.ConclusionThis review confirms a rapidly growing body of research into using ML for predicting delirium risk in hospital settings. Our findings offer insights for both developers and clinicians into strengths and limitations of current ML delirium prediction applications aiming to support but not usurp clinician decision-making.

show abstract

“…Much of the work utilizing AI in intensive care has been around predictive analytics for early identification of deteriorating/at risk patients and machine learning models to predict patient's clinical trajectory [111]. Solutions such as using predictive analytics to provide early insights on whether a patient will develop delirium [113], pressure injuries [114] from prolonged ICU stay, sepsis [115], or have unattended bed exits triggering falls, have the potential to become the gold standard for the healthcare industry. They augment the decision support process, enhance reliability, and accelerate much-needed agility in critical care.…”

Section: Decision Support Systems and Artificial Intelligencementioning

confidence: 99%

Patient Safety in the Critical Care Setting: Common Risks and Review of Evidence-Based Mitigation Strategies

Arteaga¹,

Bacu²,

Franco³

2023

Contemporary Topics in Patient Safety - Volume 2

View full text Add to dashboard Cite

The Intensive Care Unit (ICU) has evolved in the last 50 years. This evolution’s main drivers include equipment and software improvements, the patient safety movement, and a better pathophysiological understanding of critical illness. There is mounting pressure from accreditation agencies, governmental regulation, financial challenges, operational dynamics, staffing changes, and increased acuity affecting-ICU care delivery and impacting patient safety. There are higher than ever expectations to improve clinical outcomes after an intensive care stay, to enhance patient safety, to increase family involvement in decision making, and merge the multidisciplinary medical experience into an effective teamwork. Leadership focus is directed towards increasing diversity and inclusion in the workforce while enhancing psychological safety. This review addresses the common risks for patient safety in the intensive care setting and describes the changes in mindset and application of evidence-based mitigation strategies.

show abstract

Artificial intelligence predicts delirium following cardiac surgery: A case study

Cited by 13 publications

References 5 publications

Machine Learning–Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance

Machine Learning–Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance

Navigating the machine learning pipeline: a scoping review of inpatient delirium prediction models

Patient Safety in the Critical Care Setting: Common Risks and Review of Evidence-Based Mitigation Strategies

Contact Info

Product

Resources

About