Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV/AIDS

Saldanha, Ian J.; Li, Tianjing; Yang, Cui; Owczarzak, Jill; Williamson, Paula; Dickersin, Kay

doi:10.1016/j.jclinepi.2017.02.005

Cited by 28 publications

(29 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although we specified only a few outcome domains, RCTs assessed hundreds of different treatment effects within them [36]. Previous authors have argued that core outcome sets, which list the minimum outcomes to assess in research, could improve the comparability of trials [37,38] and improve the synthesis of trials for knowledge translation [39]. Although we agree, our results also show that efforts to develop core outcome sets might have little impact unless they define outcomes completely.…”

Section: Discussionsupporting

confidence: 55%

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Mayo‐Wilson

Fusco

et al. 2017

Journal of Clinical Epidemiology

Self Cite

115

163

View full text Add to dashboard Cite

Objective: To identify variations in outcomes and results across reports of randomized clinical trials (RCTs). Study Design and Setting: Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected ''outcomes'' (i.e., domain, measure, metric, method of aggregation, and time point); ''treatment effect'' (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis.Results: We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports.Conclusion: RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information. Ó 2017 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

show abstract

Section: Discussionsupporting

confidence: 55%

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Mayo‐Wilson

Fusco

et al. 2017

Journal of Clinical Epidemiology

Self Cite

115

163

View full text Add to dashboard Cite

show abstract

“…23 These findings reflect discord among reviewers and trialists addressing the same disease, in addition to the increasing evidence of the inconsistency in outcome use among trials. Other systematic investigations of trials that addressed HIV infection and AIDS, 23 tinnitus, 24 cardiac arrest, 25 and critical care 26 have also demonstrated the absence of a single outcome that was reported across all trials. The proportion of outcomes reported in only 1 trial each has been reported to be high, ranging from 41% to 70%, for HIV infection and AIDS, 23 glaucoma, 13 cardiothoracic surgery, 27 and audiology.…”

Section: Discussionmentioning

confidence: 99%

“…4 We previously tested a framework for this approach for outcomes in trials and reviews that addressed HIV infection and AIDS. 23,38 Macefield and colleagues 39 also used a similar framework while identifying patient-reported core outcomes for esophageal cancer.…”

Section: Discussionmentioning

confidence: 99%

Comparison of Clinical Trial and Systematic Review Outcomes for the 4 Most Prevalent Eye Diseases

et al. 2017

Self Cite

View full text Add to dashboard Cite

IMPORTANCE Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies.OBJECTIVES To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews. DESIGN, SETTING, AND PARTICIPANTSThis cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease. MAIN OUTCOMES AND MEASURES Proportion of review outcomes also reported in trials and vice versa.RESULTS This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews. CONCLUSIONS AND RELEVANCEAlthough most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.

show abstract

“…Consistent with guidance in the Cochrane Handbook for Systematic Reviews of Interventions [6], we believe that studies that address the population, interventions, and comparators of interest should be included and cataloged in systematic reviews even if they do not report outcomes of interest. Outcome choice in a review is crucial because: (1) outcomes serve as yardsticks for basing conclusions about treatments; and (2) which outcomes are chosen and how they are defined can impact how many meta-analyses can be done and how many studies can be included in them [7][8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

“…We previously demonstrated, through case studies in the fields of eyes and vision [11] and HIV/AIDS [10], that reviewers and clinical trialists addressing the same research question often examine different outcomes. In addition, inconsistency in outcome reporting across eligible studies prevents incorporation of all eligible studies into meta-analyses.…”

Section: Introductionmentioning

confidence: 99%

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

Saldanha¹,

Lindsley

Money

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background: There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into metaanalyses. Methods: We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome. Results: We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews). Conclusions: Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

show abstract

Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV/AIDS

Cited by 28 publications

References 26 publications

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Comparison of Clinical Trial and Systematic Review Outcomes for the 4 Most Prevalent Eye Diseases

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

Contact Info

Product

Resources

About