Empirical use of causal inference methods to evaluate survival differences in a real‐world registry vs those found in randomized clinical trials

Abstract

With heighted interest in causal inference based on real‐world evidence, this empirical study sought to understand differences between the results of observational analyses and long‐term randomized clinical trials. We hypothesized that patients deemed “eligible” for clinical trials would follow a different survival trajectory from those deemed “ineligible” and that this factor could partially explain results. In a large observational registry dataset, we estimated separate survival trajectories for hypothetically trial‐eligible vs ineligible patients under both coronary artery bypass surgery (CABG) and percutaneous coronary intervention (PCI). We also explored whether results would depend on the causal inference method (inverse probability of treatment weighting vs optimal full propensity matching) or the approach to combine propensity scores from multiple imputations (the “across” vs “within” approaches). We found that, in this registry population of PCI/CABG multivessel patients, 32.5% would have been eligible for contemporaneous RCTs, suggesting that RCTs enroll selected populations. Additionally, we found treatment selection bias with different distributions of propensity scores between PCI and CABG patients. The different methodological approaches did not result in different conclusions. Overall, trial‐eligible patients appeared to demonstrate at least marginally better survival than ineligible patients. Treatment comparisons by eligibility depended on disease severity. Among trial‐eligible three‐vessel diseased and trial‐ineligible two‐vessel diseased patients, CABG appeared to have at least a slight advantage with no treatment difference otherwise. In conclusion, our analyses suggest that RCTs enroll highly selected populations, and our findings are generally consistent with RCTs but less pronounced than major registry findings.

Trial emulation and survival analysis for disease incidence registers: A case study on the causal effect of pre‐emptive kidney transplantation

“…Here too ITT marginalizes over subsequent treatment (intensity). An appreciation of exposure levels that follow in the study population will deepen our understanding of exposure and influence transportability of the estimand 3,7,8 . Of course, before comparing outcomes of treatment groups conditional on covariates in the emulated trial, explicit adjustment for baseline confounders is required.…”

“…An appreciation of exposure levels that follow in the study population will deepen our understanding of exposure and influence transportability of the estimand. 3,7,8 Of course, before comparing outcomes of treatment groups conditional on covariates in the emulated trial, explicit adjustment for baseline confounders is required. This will ensure exchangeability before averaging over a chosen distribution of baseline covariates.…”

Real-world data and evidence in pain research: a qualitative systematic review of methods in current practice

“… 2 , 4 , 18 , 76 The fast-emerging field of causal inference develops methods designated to drawing high degrees of evidence from nonexperimental data 32 and can be especially used in RWE studies. 48 …”

“… 20 The use of rigorous causal inference methods will allow for high-level evidence to be drawn from RWD studies. 32 , 48 As is to be expected with emergent technologies, methodological improvements and increased rigor are needed. Statisticians and epidemiologists should be included from the early planning stage, preregistration as well as transparent project timelines should be mandatory, and a widely accepted standard terminology will be needed to make these works accessible and the evidence generated of high quality.…”