Emulation Differences vs. Biases When Calibrating Real‐World Evidence Findings Against Randomized Controlled Trials

Franklin, Jessica M.; Glynn, Robert J.; Suissa, Samy; Schneeweiß, Sebastian

doi:10.1002/cpt.1793

Cited by 39 publications

(36 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This requires consistency in the registration of variables by hospitals and the measurement of clinical outcomes. Important factors supporting the consistency, allowing the use of registries in medicine evaluation by regulators, would include, for example, the use of common datasets and coding terminologies, complete data collection, and quality assurance and consistent governance [ 15 , 20 ]. The registry used in this study, the DMTR, is a validated, nationwide patient registry with limited missing patients and data.…”

Section: Discussionmentioning

confidence: 99%

Postapproval trials versus patient registries: comparability of advanced melanoma patients with brain metastases

Ismail

Sikkes²,

Wouters

et al. 2020

Melanoma Research

View full text Add to dashboard Cite

Postapproval trials and patient registries have their pros and cons in the generation of postapproval data. No direct comparison between clinical outcomes of these data sources currently exists for advanced melanoma patients. We aimed to investigate whether a patient registry can complement or even replace postapproval trials. Postapproval single-arm clinical trial data from the Medicines Evaluation Board and real-world data from the Dutch Melanoma Treatment Registry were used. The study population consisted of advanced melanoma patients with brain metastases treated with targeted therapies (BRAF- or BRAF-MEK inhibitors) in the first line. A Cox hazard regression model and a propensity score matching (PSM) model were used to compare the two patient populations. Compared to patients treated in postapproval trials ( n = 467), real-world patients ( n = 602) had significantly higher age, higher ECOG performance status, more often ≥3 organ involvement and more symptomatic brain metastases. Lactate dehydrogenase levels were similar between both groups. The unadjusted median overall survival (mOS) in postapproval clinical trial patients was 8.7 (95% CI, 8.1–10.4) months compared to 7.2 (95% CI, 6.5–7.7) months ( P < 0.01) in real-world patients. With the Cox hazard regression model, survival was adjusted for prognostic factors, which led to a statistically insignificant difference in mOS for trial and real-world patients of 8.7 (95% CI, 7.9–10.4) months compared to 7.3 (95% CI, 6.3–7.9) months, respectively. The PSM model resulted in 310 matched patients with similar survival ( P = 0.9). Clinical outcomes of both data sources were similar. Registries could be a complementary data source to postapproval clinical trials to establish information on clinical outcomes in specific subpopulations.

show abstract

Section: Discussionmentioning

confidence: 99%

Postapproval trials versus patient registries: comparability of advanced melanoma patients with brain metastases

Ismail

Sikkes²,

Wouters

et al. 2020

Melanoma Research

View full text Add to dashboard Cite

show abstract

“…However, if real-world data are to be used to complement trials, distinction must be made between emulation differences arising when comparing data from different sources, and potential sources of bias which may be introduced in the real-world setting. 41 This study illustrated how variations in the selection of comparator groups in the real-world setting may impact the replicability of findings from RCTs. As we used real-world data only, there were some inherent limitations.…”

Section: Discussionmentioning

confidence: 95%

“…Indeed, the methods presented in this paper demonstrate how the use of multidimensional data sources, such as electronic heath records or claims data, may supplement clinical trial data. However, if real‐world data are to be used to complement trials, distinction must be made between emulation differences arising when comparing data from different sources, and potential sources of bias which may be introduced in the real‐world setting 41 …”

Section: Discussionmentioning

confidence: 99%

Use of Real‐World Data to Emulate a Clinical Trial and Support Regulatory Decision Making: Assessing the Impact of Temporality, Comparator Choice, and Method of Adjustment

Abrahami

Pradhan

Yin

et al. 2020

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

External controls have been primarily used in the setting of single‐arm trials of rare diseases; their use in common diseases has not been readily investigated, nor is there guidance on how to best select comparators. Thus, the objective of this study was to emulate a large cardiovascular outcome trial of type 2 diabetes to compare associations of effectiveness with different comparator groups to those reported in the trial. Using the Liraglutide Effect and Action in Diabetes: Evaluation of Cardiovascular Outcome Results (LEADER) trial, we investigated six comparator groups using three calendar time periods (Early: 1999–2003; Later: 2004–2008, and Contemporaneous: 2009–2013) and two comparators (sulfonylureas and other second‐to‐third‐line antidiabetic drugs). Hazard ratios (HRs) of the three‐point composite cardiovascular outcome were estimated using four variations of the propensity score (adjustment, stratification, fine stratification, and matching) and compared with the LEADER trial (HR, 0.87; 95% confidence interval, 0.78–0.97). When comparing users of liraglutide with users of sulfonylureas, the HRs ranged from 0.57 to 1.03, with estimates in the early period most closely reflecting the LEADER trial (HR, 0.57–0.88). In contrast, the HRs ranged from 0.73 to 0.97 when comparing liraglutide users with users of any second‐to‐third‐line antidiabetic drugs, although the later period generated estimates closest to the LEADER trial (HR, 0.77–0.84). Different methods of adjustment led to generally consistent HRs, aside from the fine stratification in the early period. This study highlights the complex interplay between comparator, temporality, and method of adjustment when selecting comparators using real‐word data. These design choices must be considered in the design of trial emulation studies.

show abstract

“…The FDA is also sponsoring research to determine the value of RWE in predicting results for randomized clinical trials that have not been completed 15 . Although there is growing evidence RWE will have value for some regulatory decisions, there are many bias‐related issues that need to be both defined and addressed 16 . The growth in RWD in data diversity, patient numbers, and data quality will have a major impact on understanding who is responding and how to use drugs more precisely.…”

Section: Six Changes Impacting Precision Dosingmentioning

confidence: 99%

Drug Dosing Recommendations for All Patients: A Roadmap for Change

Powell¹,

Cook

Wang

et al. 2020

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Most drug labels do not contain dosing recommendations for a significant portion of real-world patients for whom the drug is prescribed. Current label recommendations predominately reflect the population studied in pivotal trials that typically exclude patients who are very young or old, emaciated or morbidly obese, pregnant, or have multiple characteristics likely to influence dosing. As a result, physicians may need to guess the correct dose and regimen for these patients. It is now feasible to provide dose and regimen recommendations for these patients by integrating available scientific knowledge and by utilizing or modifying current regulatory agency-industry practices. The purpose of this commentary is to explore several factors that should be considered in creating a process that will provide more effective, safe, and timely drug dosing recommendations for most, if not all, patients. These factors include the availability of real-world data, development of predictive models, experience with the US Food and Drug Administration (FDA)'s pediatric exclusivity program, development of clinical decision software, funding mechanisms like the Prescription Drug Users Fee Act (PDUFA), and harmonization of global regulatory policies. From an examination of these factors, we recommend a relatively simple, efficient expansion of current practices designed to predict, confirm, and continuously improve drug dosing for more patients. We believe implementing these recommendations will benefit patients, payers, industry, and regulatory agencies. Today clinicians may have to guess the drug dosing regimen if the patient is a neonate, very old, morbidly obese, emaciated, or has any number of other factors (e.g., unusual genotype, renal failure, and/or taking an interacting drug). It has been estimated that clinicians treating neonate patients may face this dilemma five times daily. 1 Relatively recent regulatory changes have provided a scientific and clinical basis for dosing pediatric patients for many new drugs. 2 Yet, drug dosing is still guesswork for clinicians treating the many patients who are not well-represented in clinical trials upon which the regulatory approved label is primarily based. Current labels typically fall between one of two patient drug dosing extremes where either each patient's dose is titrated to a biomarker (e.g., plasma glucose, cholesterol, or drug concentration), or one dosing regimen is approved for all adult patients without adequately understanding the limits based on age, size, organ function, genetics, or drug interactions. At market approval, the new drug labelled dosing regimen usually reflects the dosing scheme used in the phase III pivotal studies, upon which the regulatory approval decision was based with additional dosing adjustments for pharmacokinetic (PK) and/or pharmacodynamic (PD) reasons. Dosing adjustment characteristics are either quantitative or directional (e.g., increase/decrease dose) and univariate (one factor, not combinations of factors). Although more recent legislation ...

show abstract

Emulation Differences vs. Biases When Calibrating Real‐World Evidence Findings Against Randomized Controlled Trials

Cited by 39 publications

References 9 publications

Postapproval trials versus patient registries: comparability of advanced melanoma patients with brain metastases

Postapproval trials versus patient registries: comparability of advanced melanoma patients with brain metastases

Use of Real‐World Data to Emulate a Clinical Trial and Support Regulatory Decision Making: Assessing the Impact of Temporality, Comparator Choice, and Method of Adjustment

Drug Dosing Recommendations for All Patients: A Roadmap for Change

Contact Info

Product

Resources

About