Treatment use in prognostic model research: a systematic review of cardiovascular prognostic studies

Abstract

Background: Ignoring treatments in prognostic model development or validation can affect the accuracy and transportability of models. We aim to quantify the extent to which the effects of treatment have been addressed in existing prognostic model research and provide recommendations for the handling and reporting of treatment use in future studies. Methods: We first describe how and when the use of treatments by individuals in a prognostic study can influence the development or validation of a prognostic model. We subsequently conducted a systematic review of the handling and reporting of treatment use in prognostic model studies in cardiovascular medicine. Data on treatment use (e.g. medications, surgeries, lifestyle interventions), the timing of their use, and the handling of such treatment use in the analyses were extracted and summarised. Results: Three hundred two articles were included in the review. Treatment use was not mentioned in 91 (30%) articles. One hundred forty-six (48%) reported specific information about treatment use in their studies; 78 (26%) provided information about multiple treatments. Three articles (1%) reported changes in medication use ("treatment drop-in") during follow-up. Seventy-nine articles (26%) excluded treated individuals from their analysis, 80 articles (26%) modelled treatment as an outcome, and of the 155 articles that developed a model, 86 (55%) modelled treatment use, almost exclusively at baseline, as a predictor. Conclusions: The use of treatments has been partly considered by the majority of CVD prognostic model studies. Detailed accounts including, for example, information on treatment drop-in were rare. Where relevant, the use of treatments should be considered in the analysis of prognostic model studies, particularly when a prognostic model is designed to guide the use of certain treatments and these treatments have been used by the study participants. Future prognostic model studies should clearly report the use of treatments by study participants and consider the potential impact of treatment use on the study findings.

When and how to use data from randomised trials to develop or validate prognostic models

“…Such details are often not available for all patients in observational databases. Post-baseline events (such as changes in treatment, the use of rescue drug treatments, or competing outcome events) might need to be accounted for when developing or externally validating a prognostic model, and should be reported alongside the results 2223…”

Accounting for time‐dependent treatment use when developing a prognostic model from observational data: A review of methods

“…However, in practice, some COPD patients do use SBBs, and these agents are known to improve survival (Du, Sun, Ding, Lu, & Chen, ; Etminan, Jafari, Carleton, & FitzGerald, ). All other (“background”) treatments were considered to be a part of routine care and should not bias the estimation of untreated risk (Pajouheshnia et al, ). A summary of the case study is presented in Supplemental Figure 1.…”

“…The proportional hazards assumption was assessed by visual inspection of Schoenfeld residuals, and no major violations were identified. As treatment use is typically ignored in most prognostic model development studies (Pajouheshnia et al, ), the Cox model ignoring treatment—now referred to as “Model 1”—served as a reference model.…”

“…This could have a number of negative consequences, including possible under‐treatment in future patients. Given that a contemporary yet untreated population in many cases no longer exists, several researchers have suggested that the effects of treatment should be accounted for in the analysis when developing a prognostic model (Cheong‐See et al, ; Groenwold et al, ; Liew et al, ; Liew, Doust, & Glasziou, ; Moons et al, ; Pajouheshnia et al, ; Sperrin et al, ). However, as of yet, there is no clear consensus over whether one analytical approach to deal with the effect of treatment is to be preferred over another.…”

Using marginal structural models to adjust for treatment drop‐in when developing clinical prediction models

“…To support treatment initiation decisions, the risk calculated by a CPM should apply to the patient assuming that no treatment is given . However, CPMs are typically derived using observed data where patients do receive treatment, often in a time‐dependent fashion . If time‐dependent treatments are not accounted for during CPM development, the subsequent risk predictions could be incorrect owing to misspecified covariate‐outcome associations, ie, the “treatment paradox.”…”