When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?

Abstract

Regulators consider randomized controlled trials (RCTs) as the gold standard for evaluating the safety and effectiveness of medications, but their costs, duration, and limited generalizability have caused some to look for alternatives. Real world evidence based on data collected outside of RCTs, such as registries and longitudinal healthcare databases, can sometimes substitute for RCTs, but concerns about validity have limited their impact. Greater reliance on such real world data (RWD) in regulatory decision making requires understanding why some studies fail while others succeed in producing results similar to RCTs. Key questions when considering whether RWD analyses can substitute for RCTs for regulatory decision making are WHEN one can study drug effects without randomization and HOW to implement a valid RWD analysis if one has decided to pursue that option. The WHEN is primarily driven by externalities not controlled by investigators, whereas the HOW is focused on avoiding known mistakes in RWD analyses.

Nonrandomized Real‐World Evidence to Support Regulatory Decision Making: Process for a Randomized Trial Replication Project

“…All trials were required to be large and sufficiently well‐powered to make their results reliable. Note that we did not intend to produce a representative sample of trials used for regulatory decision making, but instead sought to create a sample of trials that would be representative of the types of trials that are potentially replicable in claims data, given current knowledge …”

“…A similar protocol template will be used for all replications, but specific design elements and operational (coding) definitions will be chosen on the basis of knowledge of the trial, including exposure and outcome measurement, inclusion and exclusion criteria, and definition of the follow‐up period, as well as knowledge of the therapeutic area and likely sources of confounding. Although design and analysis will vary, depending on the specific study, whenever possible, we will utilize design features that have been shown previously to be more likely to lead to valid estimates, including new user, active comparator cohort designs . When possible and recommended, the study protocol will include several alternative analyses, clearly declaring one as the primary analysis that would be recommended as most likely to be valid.…”

Risk of amputations associated with SGLT2 inhibitors compared to DPP‐4 inhibitors: A propensity‐matched cohort study

“…[11][12][13] Potential reasons for discrepancies observed in the findings of these trials have been discussed elsewhere. [14][15][16][17] Two recently published observational studies report conflicting findings. [14][15][16][17] Two recently published observational studies report conflicting findings.…”

Comparative performance of a modified landmark approach when no time of treatment data are available within oncological databases: exemplary cohort study among resected pancreatic cancer patients

“…Firstly, the evidence of major RCTs on this question is quite strong, serving as a good estimate of the “true” treatment association by ACT. Secondly, resected PCa patients represent a small but homogeneous group of patients, which provides a better basis to study treatment associations as there are less influential factors that might distort a potential association 39. Thirdly, the ESPAC-3 trial revealed that survival of PCa patients does not depend on the time of initiation of ACT after surgery, which would have introduced bias had there been a causal relationship 18,19.…”