Development and Validation of a Quick Sepsis-Related Organ Failure Assessment-Based Machine-Learning Model for Mortality Prediction in Patients with Suspected Infection in the Emergency Department

Kwon, Young Suk; Baek, Moon Seong

doi:10.3390/jcm9030875

Cited by 21 publications

(23 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SHAP algorithm was applied to evaluate the contribution of the time dimension. The results showed that various OD types have different impacts with the respiration system, the CNS system, and the cardiovascular system ranking the top three, consistent with what are used in the popular qSOFA score [5,20]. Among all the temporary variables, the impacts of SOFA OUTI, which re ects the duration of persistent(unalleviated) OD, ranks highest, even higher than that of renal dysfunction, indicating the importance of the time dimension.…”

Section: Discussionsupporting

confidence: 66%

A Time-Incorporated SOFA-Based Explainable Machine Learning Model for Mortality Prediction in Critically Ill Patients

Liu

Gao

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Organ dysfunction (OD) assessment is essential in intensive care units (ICUs). However, no OD scoring system has so far considered the duration of OD, which is clinically relevant. This study aimed to develop and validate an ICU mortality prediction model based on the Sequential Organ Failure Assessment (SOFA) score, incorporating the time dimension with machine learning methods.Methods: Data from the eICU Collaborative Research Database and Medical Information Mart for Intensive Care (MIMIC) -III were mixed for model development, and the MIMIC-IV dataset and Nanjing Jinling Hospital Surgery ICU (SICU-JL) dataset were used for external testing. Adult patients in the ICUs for more than 72 hours were deemed eligible. The total SOFA score and individual scores were calculated every 12 hours for the first three days of ICU admission. Time-dimensional variables were derived from the consecutively recorded SOFA scores and individual scores for each organ. A modified SOFA model incorporating the time dimension (T-SOFA) was stepwise constructed to predict ICU mortality using multiple machine learning algorithms. The predictive performance was assessed with the area under the receiver operating characteristic curves (AUROC). Also, we utilized the SHapley Additive exPlanations (SHAP) algorithm for data visualization and model explainability.Results: We extracted a total of 66,709 ICU patients from the mixed datasets for model development and 15,423 patients for validation. The T-SOFA M3 that incorporated the time dimension features and age, using the XGBoost algorithm, significantly outperformed the original SOFA scores (AUROC 0.800 95% CI [0.787-0.813] vs. 0.693 95% CI [0.678-0.709], p<0.01) in the validation set. Good prediction performance was maintained for the T-SOFA M3 in test Set A and test Set B, with AUROC of 0.803, 95% CI[0.791-0.815], and 0.830, 95%CI [0.789-0.870], respectively. Significant contributors demonstrated by the SHAP analysis included total SOFA score, Respiration-score, CNS-score, age, Cardiovascular-score, and SOFA Organ dysfunction Unalleviated Time Index.Conclusions: A SOFA-based, time-incorporated prediction model was developed and validated by machine learning algorithms, showing satisfactory predictability and medical interpretability.

show abstract

Section: Discussionsupporting

confidence: 66%

A Time-Incorporated SOFA-Based Explainable Machine Learning Model for Mortality Prediction in Critically Ill Patients

Liu

Gao

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…As few as thousands to tens of thousands of samples may be required [ 31 ]. In this study, unlike previous study with same algorithms [ 32 ], it was conducted prospectively, and we tried to include the maximum amount of training data in consideration of the expected study period and the difficulty of obtaining data. After oversampling with SMOTE, each class of train set was 1173.…”

Section: Discussionmentioning

confidence: 99%

Development and validation of a difficult laryngoscopy prediction model using machine learning of neck circumference and thyromental height

Kim

Jang

et al. 2021

BMC Anesthesiol

Self Cite

View full text Add to dashboard Cite

Background Predicting difficult airway is challengeable in patients with limited airway evaluation. The aim of this study is to develop and validate a model that predicts difficult laryngoscopy by machine learning of neck circumference and thyromental height as predictors that can be used even for patients with limited airway evaluation. Methods Variables for prediction of difficulty laryngoscopy included age, sex, height, weight, body mass index, neck circumference, and thyromental distance. Difficult laryngoscopy was defined as Grade 3 and 4 by the Cormack-Lehane classification. The preanesthesia and anesthesia data of 1677 patients who had undergone general anesthesia at a single center were collected. The data set was randomly stratified into a training set (80%) and a test set (20%), with equal distribution of difficulty laryngoscopy. The training data sets were trained with five algorithms (logistic regression, multilayer perceptron, random forest, extreme gradient boosting, and light gradient boosting machine). The prediction models were validated through a test set. Results The model’s performance using random forest was best (area under receiver operating characteristic curve = 0.79 [95% confidence interval: 0.72–0.86], area under precision-recall curve = 0.32 [95% confidence interval: 0.27–0.37]). Conclusions Machine learning can predict difficult laryngoscopy through a combination of several predictors including neck circumference and thyromental height. The performance of the model can be improved with more data, a new variable and combination of models.

show abstract

“…18 19 Although several studies based on different models were conducted, no single model has yet proved to be superior regarding clinical efficacy. [20][21][22][23][24] Despite the use of vital signs in patient monitoring and clinical decision-making, the importance of specific types of vital signs, their correlation and frequency of registration to best prevent adverse events and in-hospital mortality is still unclear. 9 However, ML could be a good solution for these challenges.…”

Section: Strengths and Limitations Of This Studymentioning

confidence: 99%

Machine learning techniques for mortality prediction in emergency departments: a systematic review

et al. 2021

View full text Add to dashboard Cite

ObjectivesThis systematic review aimed to assess the performance and clinical feasibility of machine learning (ML) algorithms in prediction of in-hospital mortality for medical patients using vital signs at emergency departments (EDs).DesignA systematic review was performed.SettingThe databases including Medline (PubMed), Scopus and Embase (Ovid) were searched between 2010 and 2021, to extract published articles in English, describing ML-based models utilising vital sign variables to predict in-hospital mortality for patients admitted at EDs. Critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist was used for study planning and data extraction. The risk of bias for included papers was assessed using the prediction risk of bias assessment tool.ParticipantsAdmitted patients to the ED.Main outcome measureIn-hospital mortality.ResultsFifteen articles were included in the final review. We found that eight models including logistic regression, decision tree, K-nearest neighbours, support vector machine, gradient boosting, random forest, artificial neural networks and deep neural networks have been applied in this domain. Most studies failed to report essential main analysis steps such as data preprocessing and handling missing values. Fourteen included studies had a high risk of bias in the statistical analysis part, which could lead to poor performance in practice. Although the main aim of all studies was developing a predictive model for mortality, nine articles did not provide a time horizon for the prediction.ConclusionThis review provided an updated overview of the state-of-the-art and revealed research gaps; based on these, we provide eight recommendations for future studies to make the use of ML more feasible in practice. By following these recommendations, we expect to see more robust ML models applied in the future to help clinicians identify patient deterioration earlier.

show abstract

Development and Validation of a Quick Sepsis-Related Organ Failure Assessment-Based Machine-Learning Model for Mortality Prediction in Patients with Suspected Infection in the Emergency Department

Cited by 21 publications

References 46 publications

A Time-Incorporated SOFA-Based Explainable Machine Learning Model for Mortality Prediction in Critically Ill Patients

A Time-Incorporated SOFA-Based Explainable Machine Learning Model for Mortality Prediction in Critically Ill Patients

Development and validation of a difficult laryngoscopy prediction model using machine learning of neck circumference and thyromental height

Machine learning techniques for mortality prediction in emergency departments: a systematic review

Contact Info

Product

Resources

About