Guide to presenting clinical prediction models for use in clinical settings

Bonnett, Laura; Snell, Kym I E; Collins, Gary S.; Riley, Richard D

doi:10.1136/bmj.l737

Cited by 144 publications

(124 citation statements)

References 32 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…The validation cohort that included 145 patients was younger: age 56 years (IQR 47-68) with a higher percentage of males (65·5%) than in the developmental cohort (P<0·001). The median (IQR) time from onset of symptoms to hospital admission was no different in the developmental 10 [7][8][9][10][11][12][13][14] and validation 10 [7][8][9][10][11][12][13] cohorts. More patients had hypertension (42·6% vs 26·9%) and heart failure (4·4% vs 0.7%) in the development than validation cohorts, whilst diabetes (18·5% vs 11·7%), chronic obstructive pulmonary disease (COPD) (5·0% vs 3·4%) and others (6·4% vs 4·8) showed no statistical difference.…”

Section: Patient Cohortsmentioning

confidence: 99%

Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19

Xie

Hungerford

Chen

et al. 2020

Preprint

Self Cite

103

View full text Add to dashboard Cite

Background: COVID-19 pandemic has developed rapidly and the ability to stratify the most vulnerable patients is vital. However, routinely used severity scoring systems are often low on diagnosis, even in non-survivors. Therefore, clinical prediction models for mortality are urgently required. Methods: We developed and internally validated a multivariable logistic regression model to predict inpatient mortality in COVID-19 positive patients using data collected retrospectively from Tongji Hospital, Wuhan (299 patients). External validation was conducted using a retrospective cohort from Jinyintan Hospital, Wuhan (145 patients). Nine variables commonly measured in these acute settings were considered for model development, including age, biomarkers and comorbidities. Backwards stepwise selection and bootstrap resampling were used for model development and internal validation. We assessed discrimination via the C statistic, and calibration using calibration-in-the-large, calibration slopes and plots. Findings: The final model included age, lymphocyte count, lactate dehydrogenase and SpO2 as independent predictors of mortality. Discrimination of the model was excellent in both internal (c=0.89) and external (c=0.98) validation. Internal calibration was excellent (calibration slope=1). External validation showed some over-prediction of risk in low-risk individuals and under-prediction of risk in high-risk individuals prior to recalibration. Recalibration of the intercept and slope led to excellent performance of the model in independent data. Interpretation: COVID-19 is a new disease and behaves differently from common critical illnesses. This study provides a new prediction model to identify patients with lethal COVID-19. Its practical reliance on commonly available parameters should improve usage of limited healthcare resources and patient survival rate.

show abstract

Section: Patient Cohortsmentioning

confidence: 99%

Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19

Xie

Hungerford

Chen

et al. 2020

Preprint

Self Cite

103

View full text Add to dashboard Cite

show abstract

“…We created an Excel spreadsheet to present the WS final model as a risk prediction calculator that can be used in the clinic to provide women with an individualised estimate of their probability of pre-eclampsia [29]. We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines to report our methods and findings [26].…”

Section: Model Comparison With Nice Approachmentioning

confidence: 99%

“…The performance of the model at lower risk thresholds is shown in Table 2. At fixed 5% FPR, corresponding to a 5.3% risk threshold, the model classified 6% of women to be at high-risk of pre-eclampsia and the sensitivity was 30% (25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36). At a fixed 10% FPR, corresponding to a 3.5% risk threshold, the model classified 11% of women to be at high-risk of pre-eclampsia and the sensitivity was 40% (95% CI [35][36][37][38][39][40][41][42][43][44][45][46] Odds = Exp Y (final prediction score) Pre-eclampsia probability = Odds / (1 + Odds) A 'WS pre-eclampsia risk prediction tool' has been created as an Excel spreadsheet that can be used in the clinic to perform these calculations automatically using information entered about a woman's risk factors (Additional file 3).…”

Section: Ws Final Modelmentioning

confidence: 99%

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: a model development and validation study

Al-Rubaie

Hudson

Jenkins

et al. 2020

BMC Pregnancy Childbirth

View full text Add to dashboard Cite

Background: Guidelines recommend identifying in early pregnancy women at elevated risk of pre-eclampsia. The aim of this study was to develop and validate a pre-eclampsia risk prediction model for nulliparous women attending routine antenatal care "the Western Sydney (WS) model"; and to compare its performance with the National Institute of Health and Care Excellence (NICE) risk factor-list approach for classifying women as high-risk. Methods: This retrospective cohort study included all nulliparous women who gave birth in three public hospitals in the Western-Sydney-Local-Health-District, Australia 2011-2014. Using births from 2011 to 2012, multivariable logistic regression incorporated established maternal risk factors to develop and internally validate the WS model. The WS model was then externally validated using births from 2013 to 2014, assessing its discrimination and calibration. We fitted the final WS model for all births from 2011 to 2014, and compared its accuracy in predicting pre-eclampsia with the NICE approach. Results: Among 12,395 births to nulliparous women in 2011-2014, there were 293 (2.4%) pre-eclampsia events. The WS model included: maternal age, body mass index, ethnicity, multiple pregnancy, family history of pre-eclampsia, autoimmune disease, chronic hypertension and chronic renal disease. In the validation sample (6201 births), the model c-statistic was 0.70 (95% confidence interval 0.65-0.75). The observed:expected ratio for pre-eclampsia was 0.91, with a Hosmer-Lemeshow goodness-of-fit test p-value of 0.20. In the entire study sample of 12,395 births, 374 (3.0%) women had a WS model-estimated pre-eclampsia risk ≥8%, the pre-specified risk-threshold for considering aspirin prophylaxis. Of these, 54 (14.4%) developed pre-eclampsia (sensitivity 18% (14-23), specificity 97% (97-98)). Using the NICE approach, 1173 (9.5%) women were classified as high-risk, of which 107 (9.1%) developed preeclampsia (sensitivity 37% (31-42), specificity 91% (91-92)). The final model showed similar accuracy to the NICE approach when using lower risk-threshold of ≥4% to classify women as high-risk for pre-eclampsia. Conclusion: The WS risk model that combines readily-available maternal characteristics achieved modest performance for prediction of pre-eclampsia in nulliparous women. The model did not outperform the NICE approach, but has the advantage of providing individualised absolute risk estimates, to assist with counselling, inform decisions for further testing, and consideration of aspirin prophylaxis.

show abstract

“…We created an Excel spreadsheet to present the WS final model as a risk prediction calculator that can be used in the clinic to provide women with an individualised estimate of their probability of preeclampsia [29]. We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines to report our methods and findings [26].…”

Section: Model Comparison With Nice Approachmentioning

confidence: 99%

Section: Ws Final Modelmentioning

confidence: 99%

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: A model development and validation study

Al-Rubaie

Hudson²,

Jenkins

et al. 2019

Preprint

View full text Add to dashboard Cite

Background Guidelines recommend identifying in early pregnancy women at elevated risk of pre-eclampsia. The aim of this study was to develop and validate a pre-eclampsia risk prediction model for nulliparous women attending routine antenatal care “the Western Sydney (WS) model”; and to compare its performance with the National Institute of Health and Care Excellence (NICE) risk factor-list approach for classifying women as high-risk. Methods This retrospective cohort study included all nulliparous women who gave birth in three public hospitals in the Western-Sydney-Local-Health-District, Australia 2011-2014. Using births from 2011-2012, multivariable logistic regression incorporated established maternal risk factors to develop and internally validate the WS model. The WS model was then externally validated using births from 2013-2014, assessing its discrimination and calibration. We fitted the final WS model for all births from 2011-2014, and compared its accuracy in predicting pre-eclampsia with the NICE approach. Results Among 12,395 births to nulliparous women in 2011-2014, there were 293 (2.4%) pre-eclampsia events. The WS model included: maternal age, body mass index, ethnicity, multiple pregnancy, family history of pre-eclampsia, autoimmune disease, chronic hypertension and chronic renal disease. In the validation sample (6201 births), the model c-statistic was 0.70 (95% confidence interval 0.65–0.75). The observed:expected ratio for pre-eclampsia was 0.91, with a Hosmer-Lemeshow goodness-of-fit test p-value of 0.20. In the entire study sample of 12,395 births, 374 (3.0%) women had a WS model-estimated pre-eclampsia risk ≥8%, the pre-specified risk-threshold for considering aspirin prophylaxis. Of these, 54 (14.4%) developed pre-eclampsia (sensitivity 18% (14–23), specificity 97% (97–98)). Using the NICE approach, 1173 (9.5%) women were classified as high-risk, of which 107 (9.1%) developed pre-eclampsia (sensitivity 37% (31-42), specificity 91% (91–92)). The final model showed similar accuracy to the NICE approach when using lower risk-threshold of ≥4% to classify women as high-risk for pre-eclampsia. Conclusion The WS risk model that combines readily-available maternal characteristics achieved modest performance for prediction of pre-eclampsia in nulliparous women. The model did not outperform the NICE approach, but has the advantage of providing individualised absolute risk estimates, to assist with counselling, inform decisions for further testing, and consideration of aspirin prophylaxis.

show abstract

Guide to presenting clinical prediction models for use in clinical settings

Cited by 144 publications

References 32 publications

Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19

Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: a model development and validation study

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: A model development and validation study

Contact Info

Product

Resources

About