Sample size for binary logistic prediction models: Beyond events per variable criteria

Smeden, Maarten van; Moons, Karel G.M.; Groot, Joris A. H. de; Collins, Gary S.; Altman, Douglas G.; Eijkemans, Marinus J.C.; Reitsma, Johannes B.

doi:10.1177/0962280218784726

Cited by 436 publications

(354 citation statements)

References 68 publications

Supporting

Mentioning

341

Contrasting

Unclassified

Order By: Relevance

“…For studies validating prognostic models, there is no solid sample size recommendation, but it is recommended to consider at least the number of predictors, the total sample size and the event fraction. [42][43][44] The low number of events in our study might have . The horizontal gray line is the net benefit when all RSVinfected hospitalized adults are considered as not having the poor outcome; vertical gray line is the net benefit when all RSV-infected hospitalized adults are considered as having the poor outcome.…”

Section: Discussionmentioning

confidence: 79%

“…First, we had a limited amount of patients with the primary outcome. For studies validating prognostic models, there is no solid sample size recommendation, but it is recommended to consider at least the number of predictors, the total sample size and the event fraction . The low number of events in our study might have resulted in biased and less precise performance measures, which is also indicated by the broad CI of the reported C ‐statistics.…”

Section: Discussionmentioning

confidence: 98%

See 1 more Smart Citation

External validation and update of a prognostic model to predict mortality in hospitalized adults with RSV: A retrospective Dutch cohort study

Vos

Oosterheert

Hoepelman

et al. 2019

Journal of Medical Virology

View full text Add to dashboard Cite

Respiratory syncytial virus (RSV) causes significant mortality in hospitalized adults. Prediction of poor outcomes improves targeted management and clinical outcomes. We externally validated and updated existing models to predict poor outcome in hospitalized RSV‐infected adults. In this single center, retrospective, observational cohort study, we included hospitalized adults with respiratory tract infections (RTIs) and a positive polymerase chain reaction for RSV (A/B) on respiratory tract samples (2005‐2018). We validated existing prediction models and updated the best discriminating model by revision, recalibration, and incremental value testing. We included 192 RSV‐infected patients (median age 60.7 years, 57% male, 65% immunocompromised, and 43% with lower RTI). Sixteen patients (8%) died within 30 days. During hospitalization, 16 (8%) died, 30 (16%) were admitted to intensive care unit, 21 (11%) needed invasive mechanical ventilation, and 5 (3%) noninvasive positive pressure ventilation. Existing models performed moderately at external validation, with C‐statistics 0.6 to 0.7 and moderate calibration. Updating to a model including lower RTI, chronic pulmonary disease, temperature, confusion and urea, increased the C‐statistic to 0.76 (95% confidence interval, 0.61‐0.91) to predict in‐hospital mortality. In conclusion, existing models to predict poor prognosis among hospitalized RSV‐infected adults perform moderately at external validation. A prognostic model may help to identify and treat RSV‐infected adults at high‐risk of death.

show abstract

Section: Discussionmentioning

confidence: 79%

Section: Discussionmentioning

confidence: 98%

External validation and update of a prognostic model to predict mortality in hospitalized adults with RSV: A retrospective Dutch cohort study

Vos

Oosterheert

Hoepelman

et al. 2019

Journal of Medical Virology

View full text Add to dashboard Cite

show abstract

“…In our simulation without noise predictors, ridge MLR tended to yield models with better calibration and overall performance than lasso MLR. Though, the relative predictive performance of lasso MLR compared to ridge MLR may improve when the number of noise variables increases, as has recently been shown for binary logistic regression …”

Section: Discussionmentioning

confidence: 98%

Sample size considerations and predictive performance of multinomial logistic prediction models

Jong

Eijkemans

Calster

et al. 2019

Statistics in Medicine

Self Cite

View full text Add to dashboard Cite

Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full‐factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer.

show abstract

“…While no consensus currently exists on the sample sizes required for prognostic prediction, the required sample size depends on several factors, including the number of predictors, total sample size, and number or proportion of events 5859. Thus, large sample sizes could be needed to reliably develop a prognostic model, especially when tens or hundreds of candidate predictor variables are considered.…”

Section: Threats To the Viability Of Rct Data Usementioning

confidence: 99%

When and how to use data from randomised trials to develop or validate prognostic models

Pajouheshnia

Groenwold

Peelen

et al. 2019

BMJ

Self Cite

View full text Add to dashboard Cite

Prediction models have become an integral part of clinical practice, providing information for patients and clinicians and providing support for their shared decision making. The development and validation of prognostic prediction models requires substantial volumes of high quality information on relevant predictors and patient health outcomes. Primary data collection dedicated to prognostic model (development or validation) research could come with substantial time and costs and can be seen as a waste of resources if suitable data are already available. Randomised clinical trials are a source of high quality clinical data with a largely untapped potential for use in further research. This article addresses when and how data from a randomised clinical trial can be used additionally for prognostic model research, and provides guidance for researchers with access to trial data to evaluate the suitability of their data for the development and validation of prognostic prediction models.

show abstract

Sample size for binary logistic prediction models: Beyond events per variable criteria

Cited by 436 publications

References 68 publications

External validation and update of a prognostic model to predict mortality in hospitalized adults with RSV: A retrospective Dutch cohort study

External validation and update of a prognostic model to predict mortality in hospitalized adults with RSV: A retrospective Dutch cohort study

Sample size considerations and predictive performance of multinomial logistic prediction models

When and how to use data from randomised trials to develop or validate prognostic models

Contact Info

Product

Resources

About