Recalibrating our prediction models in the ICU: time to move from the abacus to the computer

Pirracchio, Romain; Ranzani, Otávio T.

doi:10.1007/s00134-014-3231-x

Cited by 14 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, to account for the UK validation cohort having a different prevalence of major trauma than in the Dutch studies, resulting in a difference in the baseline risk, the intercept was re-estimated (33). The intercept is the expected proportion of major trauma when all the model predictors are zero/negative.…”

Section: Discussionmentioning

confidence: 99%

External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

Shanahan

Fuller

Sheldon

et al. 2021

Injury

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

Shanahan

Fuller

Sheldon

et al. 2021

Injury

View full text Add to dashboard Cite

“…Most studies only analyze the discriminative abilities and Brier score of the ACS risk calculator. Calibration itself is at least as important as discrimination, potentially being the source of the divergence observed [8,26,31]. Moreover, this study is one of the two studies that compares the ACS Brier score to a null model Brier score [15].…”

Section: Discussionmentioning

confidence: 99%

Inaccurate Risk Assessment by the ACS NSQIP Risk Calculator in Aortic Surgery

Hers

Schaik

Keekstra

et al. 2021

JCM

View full text Add to dashboard Cite

Objectives: The aim of this retrospective study was to assess the predictive performance of the American College of Surgeons (ACS) risk calculator for aortic aneurysm repair for the patient population of a Dutch tertiary referral hospital. Methods: This retrospective study included all patients who underwent elective endovascular or open aortic aneurysm repair at our institution between the years 2013 and 2019. Preoperative patient demographics and postoperative complication data were collected, and individual risk assessments were generated using five different current procedural terminology (CPT) codes. Receiver operating characteristic (ROC) curves, calibration plots, Brier scores, and Index of Prediction Accuracy (IPA) values were generated to evaluate the predictive performance of the ACS risk calculator in terms of discrimination and calibration. Results: Two hundred thirty-four patients who underwent elective endovascular or open aortic aneurysm repair were identified. Only five out of thirteen risk predictions were found to be sufficiently discriminative. Furthermore, the ACS risk calculator showed a structurally insufficient calibration. Most Brier scores were close to 0; however, comparison to a null model though IPA-scores showed the predictions generated by the ACS risk calculator to be inaccurate. Overall, the ACS risk calculator showed a consistent underestimation of the risk of complications. Conclusions: The ACS risk calculator proved to be inaccurate within the framework of endovascular and open aortic aneurysm repair in our medical center. To minimize the effects of patient selection and cultural differences, multicenter collaboration is necessary to assess the performance of the ACS risk calculator in aortic surgery.

show abstract

“…Calibration was assessed for the validation data using calibration plots and the Hosmer-Lemeshow test. Recalibration was performed by re-estimating the intercept and slope for the model prediction in the validation cohort [25].…”

Section: Discussionmentioning

confidence: 99%

Clinical predictors of response to methotrexate in patients with rheumatoid arthritis: a machine learning approach using clinical trial data

et al. 2022

View full text Add to dashboard Cite

Background Methotrexate is the preferred initial disease-modifying antirheumatic drug (DMARD) for rheumatoid arthritis (RA). However, clinically useful tools for individualized prediction of response to methotrexate treatment in patients with RA are lacking. We aimed to identify clinical predictors of response to methotrexate in patients with rheumatoid arthritis (RA) using machine learning methods. Methods Randomized clinical trials (RCT) of patients with RA who were DMARD-naïve and randomized to placebo plus methotrexate were identified and accessed through the Clinical Study Data Request Consortium and Vivli Center for Global Clinical Research Data. Studies with available Disease Activity Score with 28-joint count and erythrocyte sedimentation rate (DAS28-ESR) at baseline and 12 and 24 weeks were included. Latent class modeling of methotrexate response was performed. The least absolute shrinkage and selection operator (LASSO) and random forests methods were used to identify predictors of response. Results A total of 775 patients from 4 RCTs were included (mean age 50 years, 80% female). Two distinct classes of patients were identified based on DAS28-ESR change over 24 weeks: “good responders” and “poor responders.” Baseline DAS28-ESR, anti-citrullinated protein antibody (ACPA), and Health Assessment Questionnaire (HAQ) score were the top predictors of good response using LASSO (area under the curve [AUC] 0.79) and random forests (AUC 0.68) in the external validation set. DAS28-ESR ≤ 7.4, ACPA positive, and HAQ ≤ 2 provided the highest likelihood of response. Among patients with 12-week DAS28-ESR > 3.2, ≥ 1 point improvement in DAS28-ESR baseline-to-12-week was predictive of achieving DAS28-ESR ≤ 3.2 at 24 weeks. Conclusions We have developed and externally validated a prediction model for response to methotrexate within 24 weeks in DMARD-naïve patients with RA, providing variably weighted clinical features and defined cutoffs for clinical decision-making.

show abstract

Recalibrating our prediction models in the ICU: time to move from the abacus to the computer

Cited by 14 publications

References 12 publications

External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

External validation of the Dutch prediction model for prehospital triage of trauma patients in South West region of England, United Kingdom

Inaccurate Risk Assessment by the ACS NSQIP Risk Calculator in Aortic Surgery

Clinical predictors of response to methotrexate in patients with rheumatoid arthritis: a machine learning approach using clinical trial data

Contact Info

Product

Resources

About