Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study

Abstract

Objective To examine how poor reporting and inadequate methods for key methodological features in randomised controlled trials (RCTs) have changed over the past three decades.Design Mapping of trials included in Cochrane reviews. Data sourcesData from RCTs included in all Cochrane reviews published between March 2011 and September 2014 reporting an evaluation of the Cochrane risk of bias items: sequence generation, allocation concealment, blinding, and incomplete outcome data. Data extractionFor each RCT, we extracted consensus on risk of bias made by the review authors and identified the primary reference to extract publication year and journal. We matched journal names with Journal Citation Reports to get 2014 impact factors. Main outcomes measuresWe considered the proportions of trials rated by review authors at unclear and high risk of bias as surrogates for poor reporting and inadequate methods, respectively. ResultsWe analysed 20 920 RCTs (from 2001 reviews) published in 3136 journals. The proportion of trials with unclear risk of bias was 48.7% for sequence generation and 57.5% for allocation concealment; the proportion of those with high risk of bias was 4.0% and 7.2%, respectively. For blinding and incomplete outcome data, 30.6% and 24.7% of trials were at unclear risk and 33.1% and 17.1% were at high risk, respectively. Higher journal impact factor was associated with a lower proportion of trials at unclear or high risk of bias. The proportion of trials at unclear risk of bias decreased over time, especially for sequence generation, which fell from 69.1% in 1986-1990 to 31.2% in 2011-14 and for allocation concealment (70.1% to 44.6%). After excluding trials at unclear risk of bias, use of inadequate methods also decreased over time: from 14.8% to 4.6% for sequence generation and from 32.7% to 11.6% for allocation concealment.Conclusions Poor reporting and inadequate methods have decreased over time, especially for sequence generation and allocation concealment. But more could be done, especially in lower impact factor journals.

Examining the theory‐effectiveness hypothesis: A systematic review of systematic reviews

“…The strengths of this overview include a focus on systematic reviews from the last decade focusing on RCT evidence (Dechartres et al ., ) in the adult population. Although several reviews have examined the theory‐effectiveness hypothesis explicitly, no overview has previously summarized this literature systematically.…”

Judgments of risk of bias associated with random sequence generation in trials included in Cochrane systematic reviews are frequently erroneous

“…The authors analyzed data from RCTs included in all CSRs published between March 2011 and September 2014, which reported an evaluation of the Cochrane RoB items, including sequence generation. The results indicate that unclear risk for sequence generation was found in 49% of trials, high risk in 4% of all trials, and low risk in 48% of trials included in analyzed CSRs [5]. Additionally, it was found that the proportion of trials with unclear RoB for sequence generation decreased over time, falling from 69% in 1986-1990 cohort of trials to 31% in 2011-2014 cohort.…”

“…A recent report analyzed the evolution of reporting and inadequate methods over time in 20920 RCTs included in Cochrane reviews [5]. The authors analyzed data from RCTs included in all CSRs published between March 2011 and September 2014, which reported an evaluation of the Cochrane RoB items, including sequence generation.…”

Network meta-analysis: the highest level of medical evidence?

“…Overall quality reporting in randomised clinical trials has improved over the past 20 years, but much more needs to be done 18. The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) group has published a statement for reporting in network meta-analyses 19.…”