Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

Abstract

BackgroundSelective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals.MethodsWe searched PubMed for randomized controlled trials from Jan 1, 2010 –Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined.Results180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results.ConclusionAcross trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

Reproducible and transparent research practices in published neurology research

“…For example, when a randomized controlled trial (RCT) contains an outcome deemed "not significant" and is selectively removed from a trial, the validity of the RCT may be questioned. Previous studies have already established outcome reporting bias as an issue within neurology, noting that only 40% of analyzed RCTs were preregistered and, therefore, prespecified their analysis [15]. This same study found outcome reporting inconsistencies that often favored statistically significant results [15].…”

“…In neurology, an audit of randomized clinical trials published in neurology journals found 180 outcome inconsistencies across 180 trials, with most inconsistencies favoring changes in accordance with statistically significant results. Additionally, only 55% of neurology trials were prospectively registered [15], providing indications that neurology researchers are not adhering to transparency and reproducibility practices early in research planning. Reproducible research and open science practices are widely endorsed by a large proportion of authors.…”

Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

“…Howard et al . assessed 180 neurology clinical trials, finding 180 major discrepancies. Findings from Rankin et al .…”

“…compared outcomes listed in trial protocols with published reports . However, a large majority of studies have used the information included in clinical trial registries, such as ClinicalTrials.gov, to evaluate the discrepancies in outcomes between the trial registration and its publication . Use of the trial registration has two primary benefits for evaluation.…”

Effect of funding source on reporting bias in studies of intravitreal anti‐vascular endothelial growth factor therapy for retinal vein occlusion

“…On the other hand, Howard and colleagues found that industry‐funded neurology studies were more likely to have improperly registered their RCT in favour of statistically significant results (Howard et al. ), and Varner discussed extensively about potential bias in the field of ophthalmology (Varner ). When Alasbali et al.…”