Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials—ESORT)

Abstract

Objective -To assess whether randomised controlled trials (RCTs) that were registered were less likely to report positive study findings compared with RCTs that were not registered and whether the association varied by funding source. Design -Cross sectional study.Study sample -All primary RCTs published in December 2012 and indexed in PubMed by November 2013. Trial registration was determined based on the report of a trial registration number in published RCTs or the identification of the trial in a search of trial registries. Trials were separated into prospectively and retrospectively registered studies.Main outcome measure -Association between trial registration and positive study findings.Results -1122 eligible RCTs were identified, of which 593 (52.9%) were registered and 529 (47.1%) were not registered. Overall, registration was marginally associated with positive study findings (adjusted risk ratio 0.87, 95% confidence interval 0.78 to 0.98), even with stratification as prospectively and retrospectively registered trials (0.87, 0.74 to 1.03 and 0.88, 0.78 to 1.00, respectively). The interaction term between overall registration and funding source was marginally statistically significant and relative risk estimates were imprecise (0.75, 0.63 to 0.89 for non-industry funded and 1.03, 0.79 to 1.36 for industry funded, P interaction=0.046). Furthermore, a statistically significant interaction was not maintained in sensitivity analyses. Within each stratum of funding source, relative risk estimates were also imprecise for the association between positive study findings and prospective and retrospective registration.Conclusion -Among published RCTs, there was little evidence of a difference in positive study findings between registered and non-registered clinical trials, even with stratification by timing of registration. Relative risk estimates were imprecise in subgroups of non-industry and industry funded trials.

Registered trials report less beneficial treatment effects than unregistered ones: a meta-epidemiological study in orthodontics

“…A recent comprehensive meta-epidemiological study reported that non-registered or retrospectively registered trials tend to show larger treatment effects [8]. Although the effect was not statistically significant, this is confirmed by cross-sectional overviews of randomized trials in cardiology [9] or general medicine [10] that report weak associations between positive findings and trial registration, although the effects were not always consistent. It is also important to note that these analyses were based on qualitative evaluations of trial results by using either a P value cut-off or the trialists' interpretation of the results in text, both of which can be problematic.…”

“…Additionally, Dechartres et al [8] found similar magnitude of bias for non-registered versus registered trials (ratio of odds ratios of 0.85)-although not statistically significant. Another cross-sectional overview of randomized trials found only a weak association between trial registration and positive study results (adjusted risk ratio of 0.87) and a possible variation between non-industry and industry funded trials [10], but concluded that evidence is inconsistent. However, it must be noted that this was a large unselected group of randomized trials across various research question and unstratified by disease, intervention, and outcome and therefore, a potentially existing effect might have been masked by the pooled analysis.…”

The Preregistration Revolution

“…For example, Kaplan and Irvin (43) observed a dramatic drop in the rate of positive results following the requirement to preregister primary outcomes in a sample of clinical trials. The benefits of preregistration are lost if researchers do not follow the preregistrations (44,45). However, there is evidence that preregistration makes it possible to detect and possibly correct selection and reporting biases (e.g., comparetrials.org/).…”

Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial

“…3 15-21 For example, a study of 1122 RCTs indexed in PubMed in December 2012 found that many did not define the primary outcome (31%), state the sample size calculation (45%) or explain the method of allocation concealment (50%). 22 This lack of transparency is a major limiting factor for readers who assess an article in order to find the answer to a specific question; it is also a major problem for scientists who perform systematic reviews and meta-analyses.…”