Combining randomized and non‐randomized evidence in clinical research: a review of methods and applications

Verde, Pablo Emilio; Ohmann, Christian

doi:10.1002/jrsm.1122

Cited by 85 publications

(79 citation statements)

References 124 publications

(225 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-design synthesis is a method that borrows information from observational data and combines it with trial data to extrapolate trial estimates to populations that were excluded from participation. 15,22 While the pivotal trial is enrolling participants, a cohort of users of the comparator treatment in the trial can be identified in observational healthcare data. When the trial is complete and ready for analysis, this cohort can be used to facilitate extrapolation of RCT findings.…”

Section: Review Of Methods To Generalize or Extrapolate Evidence Frommentioning

confidence: 99%

Using Real‐World Data to Extrapolate Evidence From Randomized Controlled Trials

Wang

Schneeweiß

Gagne

et al. 2018

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Randomized controlled trials (RCTs) provide evidence for regulatory agencies, shape clinical practice, influence formulary decisions, and have important implications for patients. However, many patient groups that are major consumers of drugs are under-represented in randomized trials. We review three methods to extrapolate evidence from trial participants to different target populations following market approval and discuss how these could be implemented in practice to support regulatory and health technology assessment decisions. Although these methods are not a substitute for less restrictive pre-approval RCTs or rigorous observational studies when sufficient data are available in the post-approval setting, they can help to fill the evidence gap that exists in the early marketing period. Early evidence using real-world data and methods for extrapolating evidence should be reported with clear explanation of assumptions and limitations especially when used to support regulatory and health technology assessment decisions.

show abstract

Section: Review Of Methods To Generalize or Extrapolate Evidence Frommentioning

confidence: 99%

Using Real‐World Data to Extrapolate Evidence From Randomized Controlled Trials

Wang

Schneeweiß

Gagne

et al. 2018

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

show abstract

“…[44][45][46][47][48][49][50][51][52][53] These articles included two comprehensive reviews 44,45 as well as individual articles, all of which were identified in the review articles. Many of the techniques described in Verde and Ohmann 44 and Doi, 45 however, have limited applicability to regenerative medicine (i.e. when only limited evidence from a small number of studies is available), as they require significant numbers of studies or data from RCTs to be applied.…”

Section: Data Sharing Statementmentioning

confidence: 99%

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Hettle

Corbett

Hinde

et al. 2017

Health Technol Assess

106

143

View full text Add to dashboard Cite

BackgroundThe National Institute for Health and Care Excellence (NICE) commissioned a ‘mock technology appraisal’ to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal.MethodsOur research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia.ResultsAn assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the ‘curative’ and ‘bridging to stem-cell transplantation’ treatment approaches separately. Within each TPP, three ‘hypothetical’ evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision.ConclusionsIt is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further.FundingThe National Institute for Health Research Health Technology Assessment programme.

show abstract

“…A gentle introduction of this area can be found in Chap. 8 of Spiegelhalter et al [2] and a recent review in Verde and Ohmann [3].…”

Section: Introductionmentioning

confidence: 95%

Two Examples of Bayesian Evidence Synthesis with the Hierarchical Meta-Regression Approach

Verde¹

2017

Bayesian Inference

View full text Add to dashboard Cite

This is the Information Age. We can expect that for a particular research question that is empirically testable, we should have a collection of evidence which indicates the best way to proceed. Unfortunately, this is not the case in several areas of empirical research and decision making. Instead, when researchers and policy makers ask a specific question, such as "What is the effectiveness of a new treatment?", the structure of the evidence available to answer this question may be complex and fragmented (e.g. published experiments may have different grades of quality, observational data, subjective judgments, etc.).Meta-analysis is a branch of statistical techniques that helps researchers to combine evidence from a multiplicity of indirect sources. A main hurdle in meta-analysis is that we not only combine results from a diversity of sources but we also combine their multiplicity of biases. Therefore, commonly applied meta-analysis methods, e.g. random-effects models, could be misleading.In this chapter we present a new method for meta-analysis that we have called: the "Hierarchical Meta-Regression" (HMR). The HMR is an integrated approach for evidence synthesis when a multiplicity of bias, coming from indirect and disparate evidence, has to be incorporated in a meta-analysis.

show abstract

Combining randomized and non‐randomized evidence in clinical research: a review of methods and applications

Cited by 85 publications

References 124 publications

Using Real‐World Data to Extrapolate Evidence From Randomized Controlled Trials

Using Real‐World Data to Extrapolate Evidence From Randomized Controlled Trials

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Two Examples of Bayesian Evidence Synthesis with the Hierarchical Meta-Regression Approach

Contact Info

Product

Resources

About