Internal validation and comparison of the prognostic performance of models based on six emergency scoring systems to predict in-hospital mortality in the emergency department

Abstract

Background Medical scoring systems are potentially useful to make optimal use of available resources. A variety of models have been developed for illness measurement and stratification of patients in Emergency Departments (EDs). This study was aimed to compare the predictive performance of the following six scoring systems: Simple Clinical Score (SCS), Worthing physiological Score (WPS), Rapid Acute Physiology Score (RAPS), Rapid Emergency Medicine Score (REMS), Modified Early Warning Score (MEWS), and Routine Laboratory Data (RLD) to predict in-hospital mortality. Methods A prospective single-center observational study was conducted from March 2016 to March 2017 in Edalatian ED in Emam Reza Hospital, located in the northeast of Iran. All variables needed to calculate the models were recorded at the time of admission and logistic regression was used to develop the models’ prediction probabilities. The Area Under the Curve for Receiver Operating Characteristic (AUC-ROC) and Precision-Recall curves (AUC-PR), Brier Score (BS), and calibration plots were used to assess the models’ performance. Internal validation was obtained by 1000 bootstrap samples. Pairwise comparison of AUC-ROC was based on the DeLong test. Results A total of 2205 patients participated in this study with a mean age of 61.8 ± 18.5 years. About 19% of the patients died in the hospital. Approximately 53% of the participants were male. The discrimination ability of SCS, WPS, RAPS, REMS, MEWS, and RLD methods were 0.714, 0.727, 0.661, 0.678, 0.698, and 0.656, respectively. Additionally, the AUC-PR of SCS, WPS, RAPS, REMS, EWS, and RLD were 0.39, 0.42, 0.35, 0.34, 0.36, and 0.33 respectively. Moreover, BS was 0.1459 for SCS, 0.1713 for WPS, 0.0908 for RAPS, 0.1044 for REMS, 0.1158 for MEWS, and 0.073 for RLD. Results of pairwise comparison which was performed for all models revealed that there was no significant difference between the SCS and WPS. The calibration plots demonstrated a relatively good concordance between the actual and predicted probability of non-survival for the SCS and WPS models. Conclusion Both SCS and WPS demonstrated fair discrimination and good calibration, which were superior to the other models. Further recalibration is however still required to improve the predictive performance of all available models and their use in clinical practice is still unwarranted.

Scoring Systems to Evaluate the Mortality Risk of Patients with Emphysematous Cystitis: A Retrospective Observational Study

“…It is also widely used to predict the mortality risk for patients with community-acquired bacteremia in Taiwan. In a recent study, the REMS score was applied to patients with COVID-19 and influenced its risk stratification [ 34 , 35 , 36 , 37 , 38 ]. Olsson et al created the REMS score in 2004 and the parameters were listed in Table 7 .…”

Development of Nomogram for Predicting Axillary Pathologic Complete Response after Neoadjuvant Therapy in Breast Cancer Patients without Distant Metastasis

“…The variables showing significance in the multivariable regression analysis were used to develop a nomogram-based model for predicting apCR status after neoadjuvant treatment in breast cancer patients without distant metastasis (M0). The area under the curve (AUC) of ROC is a discrimination measure which represents the ability of the model to assign higher probability of apCR patients than non-apCR patients [16]. The yield values were from 0.5 (no predictive power) to 1.0 (perfect prediction).…”

Artificial intelligence and machine learning for hemorrhagic trauma care

“…Trauma and injury scoring systems can be crucial for injury characterization, especially in terms of assessing and providing prognosis for trauma incidents [ 116 ]. While current scoring systems are substandard, triaging departments often utilize them to evaluate patients efficiently by separating them on the degree of injury and threat of mortality and/or morbidity [ 117 ]. This presents an inherent standard for the measurement of trauma and/or injury as well as for making accurate prognoses.…”