Methods for individual participant data meta-analysis in prediction research

Jong, Valentijn M. T. de

doi:10.33540/469

Cited by 1 publication

(1 citation statement)

References 351 publications

(722 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have recently suggested to use propensity scores to assist external validation of clinical prediction models. 28,29 This article explores the untapped value of propensity-based standardization in the validation of clinical prediction models using IPD from two or more sources (studies or another form of clusters). We aim to provide metrics of discriminative ability that are adjusted for case-mix variation, and to test statistical properties of this methodology.…”

Section: Introductionmentioning

confidence: 99%

Propensity‐based standardization to enhance the validation and interpretation of prediction model discrimination for a target population

Jong

Hoogland

Moons

et al. 2023

Statistics in Medicine

View full text Add to dashboard Cite

External validation of the discriminative ability of prediction models is of key importance. However, the interpretation of such evaluations is challenging, as the ability to discriminate depends on both the sample characteristics (ie, case-mix) and the generalizability of predictor coefficients, but most discrimination indices do not provide any insight into their respective contributions.To disentangle differences in discriminative ability across external validation samples due to a lack of model generalizability from differences in sample characteristics, we propose propensity-weighted measures of discrimination. These weighted metrics, which are derived from propensity scores for sample membership, are standardized for case-mix differences between the model development and validation samples, allowing for a fair comparison of discriminative ability in terms of model characteristics in a target population of interest. We illustrate our methods with the validation of eight prediction models for deep vein thrombosis in 12 external validation data sets and assess our methods in a simulation study. In the illustrative example, propensity score standardization reduced between-study heterogeneity of discrimination, indicating that between-study variability was partially attributable to case-mix. The simulation study showed that only flexible propensity-score methods (allowing for non-linear effects) produced unbiased estimates of model discrimination in the target population, and only when the positivity assumption was met. Propensity score-based standardization may facilitate the interpretation of (heterogeneity in) discriminative ability of a prediction model as observed across multiple studies, and may guide model updating strategies for a particular target population. Careful propensity score modeling with attention for non-linear relations is recommended.

show abstract

Section: Introductionmentioning

confidence: 99%