Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: A randomized controlled trial

Abstract

This randomized controlled trial (RCT) aimed to test the efficacy of enhanced access to Cochrane Handbook (Handbook) recommendations for judging the 2011 Cochrane risk of bias (RoB) domains for improving the adequacy of RoB judgments. Parallel‐group RCT with a 1:1 allocation ratio (N = 2271 per group) was conducted. Eligible participants were corresponding authors of all published Cochrane reviews and protocols. After allocation by a random number generator, participants received 20 scenarios for assessing RoB. The intervention group was shown tables from the Handbook with instructions for assessing 2011 RoB tool together with scenarios they were supposed to assess—enhanced access to the Handbook. The control group was shown only a general link to the Handbook. The primary outcome was the proportion of participants that made an adequate judgment of RoB scenarios for analyzed domains. There were 240 responses out of 2020 delivered e‐mail invitations in the intervention and 197/2254 in the control group. Only five participants from the intervention group judged RoB adequately in all the 20 scenarios and no one in the control group. The proportion of participants who adequately assessed all the scenarios within a domain was significantly higher in the intervention than in the control group. The frequency of adequate RoB judgments was 7.1% (95% CI: 5.0–9.3%, p < 0.001) higher in the intervention group (76.2%) than in the control group (69.0%). The enhanced access yields more adequate RoB assessments and could be incorporated in software supporting the RoB tool.

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non‐Cochrane systematic reviews published in 2020: Meta‐research study

“…22 Our intervention achieved modest results, as the frequency of adequate RoB judgments was 7.1% higher in the intervention group (76.2%) than in the control group (69.0%). 22 However, even modest improvements on a large scale of systematic review production will achieve some effects. The enhanced access yields more adequate RoB assessments and could be incorporated into software supporting the RoB tool.…”

“…The enhanced access yields more adequate RoB assessments and could be incorporated into software supporting the RoB tool. 22 Furthermore, this intervention can be combined with others to achieve better results.…”

Dyadic psycho‐social interventions for stroke survivors and family caregivers: A systematic review and meta‐analysis of randomized controlled trials

“…Two investigators independently evaluated the quality of the literature using the Cochrane risk assessment tool (Barcot et al, 2021), which included six parts (selection bias, implementation bias, measurement bias, follow-up bias, reporting bias and other biases) with each aspect rated as low, high, or unclear in the risk of bias and then crosschecked the results. When there was any disagreement, they discussed the results with a third investigator to reach a consensus.…”

Effectiveness of red yeast rice on carotid atherosclerosis: A systematic review and meta-analysis

“…Quality assessment of the RCTs was assessed based on the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) (Sterne et al, 2019). In this tool, the study quality was have: 1) a low risk of bias, if a low risk of bias was observed in all domains; 2) a moderate risk of bias, if some concerns were present in at least one domain; or 3) a high risk of bias, if a high risk of bias was observed in at least one domain or some concerns were present for multiple domains such that the confidence in the result was substantially decreased (Sterne et al, 2019;Barcot et al, 2021).…”