Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent

Babić, Andrija; Vuka, Ivana; Saric, Frano; Prološčić, Ivona; Slapnicar, Ema; Cavar, Jakica; Peričić, Tina Poklepović; Pieper, Dawid; Puljak, Livia

doi:10.1016/j.jclinepi.2019.11.008

Cited by 21 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, RoB assessments made by authors of published systematic reviews should not be taken at the face value, as we have shown in multiple studies that RoB assessments in many Cochrane reviews were inadequate and inconsistent [7][8][9][10][11][12][13][14]. Due to the specificities of surgical trials, we hypothesized that assessments of surgical trials may be more accurate and more consistent, compared to RCTs of nonsurgical interventions.…”

Section: Introductionmentioning

confidence: 99%

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Barčot

Borić

Došenović

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background Bias in randomized controlled trials (RCTs) can lead to underestimation or overestimation of the true effects of interventions. Surgical RCTs may suffer from the risk of bias (RoB) that is avoidable in trials of other interventions, and vice versa. We aimed to compare the adequacy of RoB assessments in surgical versus non-surgical RCTs included in Cochrane reviews and to assess the most common differences in those RoB assessments. Due to specificities of surgical trials, i.e. difficulties associated with blinding of surgical interventions, we hypothesized that assessments of surgical trials may be more adequate, compared to RCTs of non-surgical interventions. Methods This was a methodological study, analyzing methods of published Cochrane systematic reviews. Data were extracted from RoB tables in Cochrane reviews (judgments and accompanying explanatory comment) for the following four RoB domains used in the 2011 Cochrane RoB tool: randomization, allocation concealment, blinding of participants and personnel, and blinding of outcome assessors. We defined adequate assessments as those that were in line with instructions from the Cochrane Handbook for Systematic Reviews of Interventions. The prevalence of adequate assessments was compared in surgical versus non-surgical trials. The most common differences in both groups of reviews were presented. Results In 729 analyzed Cochrane reviews, there were 10,537 included trials. The prevalence of adequate RoB judgments made by Cochrane authors ranged from 87.9, 95%CI (87.3 to 88.6%) for randomization to 70.7, 95%CI (69.8 to 71.5%) for blinding of participants and personnel. For all analyzed RoB domains, the prevalence of adequate RoB domains was higher in surgical trials than in non-surgical trials. For two RoB domains assessing blinding, this difference between surgical and non-surgical trials was statistically significant (P < 0.001), while the difference was not significant for the RoB domain regarding randomization (P = 0.124) and allocation concealment (P = 0.039, β < 0.8). Conclusions RoB judgments were more in line with instructions from the Cochrane Handbook when Cochrane reviews assessed surgical trials, compared to those that analyzed non-surgical interventions. However, further steps are warranted to scrutinize RoB assessment in trials of both surgical and non-surgical interventions.

show abstract

Section: Introductionmentioning

confidence: 99%

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Barčot

Borić

Došenović

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

show abstract

“…After extending the search and running new analysis, we obtained new results, and the number of analyzed reviews doubled. However, there was no difference in our results compared with the original analysis that included reviews published within 1 year only [9]. We have expected this result because we did not consider that there were any events in the meantime that would account for different results for a 3-year period compared with a 1-year period.…”

mentioning

confidence: 66%

“…Extending search dates to more years could show temporal trends in observed variables, but this is not necessarily the case, and expanding the search period usually involves a lot of additional effort and manpower. Recently, we have had such experience with an MS in which we analyzed methods for risk of bias assessment in Cochrane reviews published in 2015 and 2016 [9]. The analysis originally included 768 Cochrane reviews.…”

mentioning

confidence: 99%

Limiting the search period in methodological studies

Puljak

Babić

Pieper

2020

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

“…In fact, some methodological studies attempt to capture both "quality of conduct" and "quality of reporting", such as Richie et al, who reported on the risk of bias in randomized trials of pharmacy practice interventions [80]. Babic et al investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [81]. Further, biases related to choice of outcomes can also be explored.…”

Section: What Is the Aim?mentioning

confidence: 99%

A tutorial on methodological studies: the what, when, how and why

Mbuagbaw

Lawson

Puljak

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste. Main body We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies? Conclusion Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

show abstract

Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent

Cited by 21 publications

References 15 publications

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Limiting the search period in methodological studies

A tutorial on methodological studies: the what, when, how and why

Contact Info

Product

Resources

About