A systematic literature review on the efficacy–effectiveness gap: comparison of randomized controlled trials and observational studies of glucose-lowering drugs

Abstract

AimTo identify a potential efficacy–effectiveness gap and possible explanations (drivers of effectiveness) for differences between results of randomized controlled trials (RCTs) and observational studies investigating glucose-lowering drugs.MethodsA systematic literature review was conducted in English language articles published between 1 January, 2000 and 31 January, 2015 describing either RCTs or observational studies comparing glucagon-like peptide-1 analogs (GLP-1) with insulin or comparing dipeptidyl peptidase-4 inhibitors (DPP-4i) with sulfonylurea, all with change in glycated hemoglobin (HbA1c) as outcome. Medline, Embase, Current Content, and Biosis were searched. Information on effect estimates, baseline characteristics of the study population, publication year, study duration, and number of patients, and for observational studies, characteristics related to confounding adjustment and selection- and information bias were extracted.ResultsFrom 312 hits, 11 RCTs and 7 observational studies comparing GLP-1 with insulin, and from 474 hits, 16 RCTs and 4 observational studies comparing DPP-4i with sulfonylurea were finally included. No differences were observed in baseline characteristics of the study populations (age, sex, body mass index, time since diagnosis of type 2 diabetes mellitus, and HbA1c) or effect sizes across study designs. Mean effect sizes ranged from −0.43 to 0.91 and from −0.80 to 1.13 in RCTs and observational studies, respectively, comparing GLP-1 with insulin, and from −0.13 to 2.70 and −0.20 to 0.30 in RCTs and observational studies, respectively, comparing DPP-4i and sulfonylurea. Generally, the identified observational studies held potential flaws with regard to confounding adjustment and selection- and information bias.ConclusionsNeither potential drivers of effectiveness nor an efficacy–effectiveness gap were identified. However, the limited number of studies and potential problems with confounding adjustment, selection- and information bias in the observational studies, may have hidden a true efficacy-effectiveness gap.

Retrospective Cohort Study: Ledipasvir-Sofosbuvir With/Without Ribavirin for Chronic Hepatitis C Post–Liver Transplant in a Real-World Population

“…This is described as the "efficacy-effectiveness gap" and may be attributed to the strict inclusion/ exclusion criteria used for clinical trials and lack of external validity. 15 While it is challenging to draw a true comparison, SVR rates from the 12-week group in the present study were high and do not appear to be different from rates achieved in the SOLAR-1 trial. Patients evaluated at UWMC in the 12-week group may have been a healthier population compared to those assessed in the SOLAR-1 trial as there were no patients with decompensated cirrhosis, while 26% of patients in the SOLAR-1 trial had this There are several limitations to this study.…”

Use of the PRECIS-II instrument to categorize reports along the efficacy-effectiveness spectrum in an hepatitis C virus care continuum systematic review and meta-analysis

“…A systematic review by Ankarfeldt, et al explored a potential efficacy-effectiveness gap as a driver of the differences in observed impact of glucose-lowering drugs between observational studies and randomized controlled trials. [5] They dichotomized papers as either being pure randomized controlled trials reflecting efficacy designs or observational studies reflecting effectiveness designs and compared mean observed effect sizes in each group. Although, Ankarfeldt, et al did not identify an efficacy-effectiveness gap (possibly as a result of limited sample size and confounding), significant gaps between efficacy and effectiveness outcome have been observed for many interventions.…”

Patient-Level Effectiveness Prediction Modeling for Glioblastoma Using Classification Trees

“…However, specific details on the mechanisms that drive individual response to temozolomide treatment in clinical practice, or on the drivers of real-world patient-level treatment effectiveness, are unknown (van Genugten et al, 2010;Eichler et al, 2011;Liu et al, 2016). To study these personal responses, traditional cohort-oriented methods, such as the Kaplan-Meier survival techniques currently used in pharmacoepidemiology (Strom and Kimmel, 2006) for investigating real-world evidence (RWE) data, have shown to be inadequate because of their difficulties to cope with heterogeneous patient populations; their restrictive assumptions regarding linear relationships among variables; their inability to provide patient-level predictions; and their inability to infer causality (Ankarfeldt et al, 2017;Arora et al, 2019).…”