Validation of the Norwegian survival prediction model in trauma (NORMIT) in Swedish trauma populations

Ghorbani, Poya; Troëng, Thomas; Brattström, Olof; Ringdal, Kjetil Gorseth; Eken, Torsten; Ekbom, Anders; Strömmer, Lisa

doi:10.1002/bjs.11306

Cited by 4 publications

(4 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A porting of NORMIT to a Finish trauma population showed a ROC-AUC of 0.83 (8) and ROC-AUC ranging 0.93-0.98 porting NORMIT 1 and 2 to a Swedish trauma population (35). Our TQIPID-based model achieved 0.971 ROC-AUC when porting, thus indicating similar performance.…”

Section: Modelsupporting

confidence: 52%

Assessing optimal methods for transferring machine learning models to low-volume and imbalanced clinical datasets: experiences from predicting outcomes of Danish trauma patients

Millarch,

Bonde,

Bonde

et al. 2023

Front. Digit. Health

View full text Add to dashboard Cite

IntroductionAccurately predicting patient outcomes is crucial for improving healthcare delivery, but large-scale risk prediction models are often developed and tested on specific datasets where clinical parameters and outcomes may not fully reflect local clinical settings. Where this is the case, whether to opt for de-novo training of prediction models on local datasets, direct porting of externally trained models, or a transfer learning approach is not well studied, and constitutes the focus of this study. Using the clinical challenge of predicting mortality and hospital length of stay on a Danish trauma dataset, we hypothesized that a transfer learning approach of models trained on large external datasets would provide optimal prediction results compared to de-novo training on sparse but local datasets or directly porting externally trained models.MethodsUsing an external dataset of trauma patients from the US Trauma Quality Improvement Program (TQIP) and a local dataset aggregated from the Danish Trauma Database (DTD) enriched with Electronic Health Record data, we tested a range of model-level approaches focused on predicting trauma mortality and hospital length of stay on DTD data. Modeling approaches included de-novo training of models on DTD data, direct porting of models trained on TQIP data to the DTD, and a transfer learning approach by training a model on TQIP data with subsequent transfer and retraining on DTD data. Furthermore, data-level approaches, including mixed dataset training and methods countering imbalanced outcomes (e.g., low mortality rates), were also tested.ResultsUsing a neural network trained on a mixed dataset consisting of a subset of TQIP and DTD, with class weighting and transfer learning (retraining on DTD), we achieved excellent results in predicting mortality, with a ROC-AUC of 0.988 and an F2-score of 0.866. The best-performing models for predicting long-term hospitalization were trained only on local data, achieving an ROC-AUC of 0.890 and an F1-score of 0.897, although only marginally better than alternative approaches.ConclusionOur results suggest that when assessing the optimal modeling approach, it is important to have domain knowledge of how incidence rates and workflows compare between hospital systems and datasets where models are trained. Including data from other health-care systems is particularly beneficial when outcomes are suffering from class imbalance and low incidence. Scenarios where outcomes are not directly comparable are best addressed through either de-novo local training or a transfer learning approach.

show abstract

Section: Modelsupporting

confidence: 52%

Assessing optimal methods for transferring machine learning models to low-volume and imbalanced clinical datasets: experiences from predicting outcomes of Danish trauma patients

Millarch,

Bonde,

Bonde

et al. 2023

Front. Digit. Health

View full text Add to dashboard Cite

show abstract

“…Our findings and conclusion contrast Norwegian models (Norwegian survival prediction model in trauma [NORMIT and NORMIT 2]), which include ASA-PS, which have outperformed TRISS in mortality prediction. 18,[29][30][31] However, these models have only been validated in Finland and Sweden, and differences in trauma patient populations, the patterns of injury (eg, penetrating trauma), and prevalence of comorbidities may explain the discrepancies compared to our United States study population. The Norwegian study reported a penetrating injury rate of 9.1% and an ASA-PS score of 1 in 74% of patients, compared to 27.1% and 19% in this study, respectively.…”

Section: Discussionmentioning

confidence: 90%

A Prospective Multicenter Comparison of Trauma and Injury Severity Score, American Society of Anesthesiologists Physical Status, and National Surgical Quality Improvement Program Calculator’s Ability to Predict Operative Trauma Outcomes

Yeates,

Nahmias,

Gabriel

et al. 2023

Anesthesia &Amp; Analgesia

View full text Add to dashboard Cite

BACKGROUND: Trauma outcome prediction models have traditionally relied upon patient injury and physiologic data (eg, Trauma and Injury Severity Score [TRISS]) without accounting for comorbidities. We sought to prospectively evaluate the role of the American Society of Anesthesiologists physical status (ASA-PS) score and the National Surgical Quality Improvement Program Surgical Risk-Calculator (NSQIP-SRC), which are measurements of comorbidities, in the prediction of trauma outcomes, hypothesizing that they will improve the predictive ability for mortality, hospital length of stay (LOS), and complications compared to TRISS alone in trauma patients undergoing surgery within 24 hours. METHODS: A prospective, observational multicenter study (9/2018–2/2020) of trauma patients ≥18 years undergoing operation within 24 hours of admission was performed. Multiple logistic regression was used to create models predicting mortality utilizing the variables within TRISS, ASA-PS, and NSQIP-SRC, respectively. Linear regression was used to create models predicting LOS and negative binomial regression to create models predicting complications. RESULTS: From 4 level I trauma centers, 1213 patients were included. The Brier Score for each model predicting mortality was found to improve accuracy in the following order: 0.0370 for ASA-PS, 0.0355 for NSQIP-SRC, 0.0301 for TRISS, 0.0291 for TRISS+ASA-PS, and 0.0234 for TRISS+NSQIP-SRC. However, when comparing TRISS alone to TRISS+ASA-PS (P = .082) and TRISS+NSQIP-SRC (P = .394), there was no significant improvement in mortality prediction. NSQIP-SRC more accurately predicted both LOS and complications compared to TRISS and ASA-PS. CONCLUSIONS: TRISS predicts mortality better than ASA-PS and NSQIP-SRC in trauma patients undergoing surgery within 24 hours. The TRISS mortality predictive ability is not improved when combined with ASA-PS or NSQIP-SRC. However, NSQIP-SRC was the most accurate predictor of LOS and complications.

show abstract

“…The inclusion criteria in SweTrau have been described previously 6 . The registry is based on the revised Utstein Trauma Template, which is the current European core dataset 17 , and has registered patients in Sweden since 2011.…”

Section: Methodsmentioning

confidence: 99%

“…RAM has become the standard method for measuring and comparing hospital performance in trauma populations, and is widely used in North America and Europe 2–4 . The Norwegian survival prediction model in trauma (NORMIT) was developed from a single level 1 trauma centre population in Norway, and externally validated in Finnish 5 and Swedish (NORMIT and the recently updated NORMIT 2) 6 trauma populations. The NORMIT models demonstrated adequate prediction of mortality in Swedish trauma centre populations but performed poorly in mixed populations of patients admitted to all hospital types 5 , 6 .…”

Section: Introductionmentioning

confidence: 99%

Risk-adjusted mortality in severely injured adult trauma patients in Sweden

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background Risk-adjusted mortality (RAM) analysis and comparisons of clinically relevant subsets of trauma patients allow hospitals to assess performance in different processes of care. The aim of the study was to develop a RAM model and compare RAM ratio (RAMR) in subsets of severely injured adult patients treated in university hospitals (UHs) and emergency hospitals (EHs) in Sweden. Methods This was a retrospective study of the Swedish trauma registry data (2013 to 2017) comparing RAMR in patients (aged 15 years or older and New Injury Severity Score (NISS) of more than 15) in the total population (TP) and in multisystem blunt (MB), truncal penetrating (PEN), and severe traumatic brain injury (STBI) subsets treated in UHs and EHs. The RAM model included the variables age, NISS, ASA Physical Status Classification System Score, and physiology on arrival. Results In total, 6690 patients were included in the study (4485 from UHs and 2205 from EHs). The logistic regression model showed a good fit. RAMR was 4.0, 3.8, 7.4, and 8.5 percentage points lower in UH versus EH for TP (P < 0.001), MB (P < 0.001), PEN (P = 0.096), and STBI (P = 0.005), respectively. The TP and MB subsets were subgrouped in with (+) and without (−) traumatic brain injury (TBI). RAMR was 7.5 and 7.0, respectively, percentage points lower in UHs than in EHs in TP + TBI and MB + TBI (both P < 0.001). In the TP–TBI (P = 0.027) and MB–TBI (P = 0.107) subsets the RAMR was 1.6 and 1.8 percentage points lower, respectively. Conclusion The lower RAMR in UHs versus EH were due to differences in TBI-related mortality. No evidence supported that Swedish EHs provide inferior quality of care for trauma patients without TBI or for patients with penetrating injuries.

show abstract

Validation of the Norwegian survival prediction model in trauma (NORMIT) in Swedish trauma populations

Cited by 4 publications

References 52 publications

Assessing optimal methods for transferring machine learning models to low-volume and imbalanced clinical datasets: experiences from predicting outcomes of Danish trauma patients

Assessing optimal methods for transferring machine learning models to low-volume and imbalanced clinical datasets: experiences from predicting outcomes of Danish trauma patients

A Prospective Multicenter Comparison of Trauma and Injury Severity Score, American Society of Anesthesiologists Physical Status, and National Surgical Quality Improvement Program Calculator’s Ability to Predict Operative Trauma Outcomes

Risk-adjusted mortality in severely injured adult trauma patients in Sweden

Contact Info

Product

Resources

About