Comparison of Clinical Trial and Systematic Review Outcomes for the 4 Most Prevalent Eye Diseases

Abstract

IMPORTANCE Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies.OBJECTIVES To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews. DESIGN, SETTING, AND PARTICIPANTSThis cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease. MAIN OUTCOMES AND MEASURES Proportion of review outcomes also reported in trials and vice versa.RESULTS This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews. CONCLUSIONS AND RELEVANCEAlthough most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

“…This suggests that, approximately 7 in 10 studies that reviewers include are not incorporated into the meta-analysis for the main outcome, and 1 in 4 studies are not incorporated into the meta-analysis for any outcome. In previous work, we demonstrated poor overlap between outcomes in clinical trials and reviews, and possible differences in the types of outcomes they examine [10,11]. For HIV/AIDS, we demonstrated that reviewers examined more long-term clinical outcomes and patient-centered outcomes than did clinical trialists.…”

“…We [10,11] and others [20] have argued that, as stakeholders in a given field, systematic reviewers should both participate in the development of and adopt core outcome sets for that field. By broadening the participation in outcome prioritization efforts, this could potentially help ensure that the outcomes that are measured and reported in research are widely relevant and important.…”

“…Consistent with guidance in the Cochrane Handbook for Systematic Reviews of Interventions [6], we believe that studies that address the population, interventions, and comparators of interest should be included and cataloged in systematic reviews even if they do not report outcomes of interest. Outcome choice in a review is crucial because: (1) outcomes serve as yardsticks for basing conclusions about treatments; and (2) which outcomes are chosen and how they are defined can impact how many meta-analyses can be done and how many studies can be included in them [7][8][9][10][11].…”

“…We previously demonstrated, through case studies in the fields of eyes and vision [11] and HIV/AIDS [10], that reviewers and clinical trialists addressing the same research question often examine different outcomes. In addition, inconsistency in outcome reporting across eligible studies prevents incorporation of all eligible studies into meta-analyses.…”

Cochrane Eyes and Vision: a perspective introducing Cochrane Corner in Eye

“…This research has engaged clinical partners and addresses topics highly relevant to producing high quality reviews. Investigators at Cochrane Eyes and Vision US satellite have published methods papers on prioritizing reviews [13][14][15][16], practice guidelines [17][18][19], identifying clinical trials [20], strengths and limitations of data sources [21][22][23][24], and outcomes in Eyes and Vision trials and reviews [25][26][27][28][29], dissemination by co-publication [30], and network meta-analysis methods as applied to open angle glaucoma [31 32].…”