On the importance of a clear definition of time horizon for time-to-event dynamic predictions: a systematic review and a concrete illustration in kidney transplantation

Abstract

Background. Time-to-event dynamic predictions are defined as the probability to survive until a defined time horizon given being event-free at landmark times and given available predictive variables at such prediction times. From two different mathematical formulations, dynamic predictions can either predict the survival probability until a final time horizon or until the end of a sliding horizon window. We aim to illustrate the need to clearly define the time horizon to correctly interpret the prognostic performances. Methods. First, following the PRISMA, CHARMS and TRIPOD recommendations, we conducted a systematic review of articles concerning dynamic predictions to assess how the time horizon was reported in the literature. Second, using a sample of 2,523 kidney recipients, we assessed the prognostic capacities of the Dynamic predictions of Patient and kidney Graft survival (DynPG) using either a final time horizon or a sliding horizon window. Results. Of 172 references retrieved about dynamic predictions, 102 articles were included in the systematic review. We notably observed that 71 (69.6%) used a sliding horizon window to assess the prognostic performance while 18 (17.7%) used a final time horizon. We also identified 13 articles (12.7%) where the time horizon was not defined clearly (or at all). Our concrete application in kidney transplantation shows that discrimination and calibration are not the same when comparing the two time horizon definitions. On one hand, for a 5-year sliding horizon window, the discrimination slightly increased as the landmark times increased, and we also observed that DynPG is reasonably well calibrated, particularly for the earliest landmark times. On the other hand, for an 11-year final time horizon, the discrimination was high for the earliest landmark times and increased over time, while the calibration plot revealed predictions were underestimated for the earliest landmark times and overestimated for later ones. Conclusions. Our systematic review identified a clear heterogeneity in the time horizon definition used, and an absence of a clear time horizon definition in a part of published articles. Our study advocates for improving the reporting when studying dynamic prediction scoring systems since the prognostic performances and interpretation differ according to the time horizon definition.