Accounting for time dependency in meta‐analyses of concordance probability estimates

Abstract

Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small‐sized validation studies. It is therefore necessary to synthesize the results of these studies using techniques for meta‐analysis. Here we consider strategies for meta analyzing the concordance probability for time‐to‐event data (“C‐index”), which has become a popular tool to evaluate the discriminatory power of prediction models with a right‐censored outcome. We show that standard meta‐analysis of the C‐index may lead to biased results, as the magnitude of the concordance probability depends on the length of the time interval used for evaluation (defined e.g., by the follow‐up time, which might differ considerably between studies). To address this issue, we propose a set of methods for random‐effects meta‐regression that incorporate time directly as covariate in the model equation. In addition to analyzing nonlinear time trends via fractional polynomial, spline, and exponential decay models, we provide recommendations on suitable transformations of the C‐index before meta‐regression. Our results suggest that the C‐index is best meta‐analyzed using fractional polynomial meta‐regression with logit‐transformed C‐index values. Classical random‐effects meta‐analysis (not considering time as covariate) is demonstrated to be a suitable alternative when follow‐up times are small. Our findings have implications for the reporting of C‐index values in future studies, which should include information on the length of the time interval underlying the calculations.