In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance

Propadalo, Ivana; Tranfic, Mia; Vuka, Ivana; Barčot, Ognjen; Peričić, Tina Poklepović; Puljak, Livia

doi:10.1016/j.jclinepi.2018.10.002

Cited by 50 publications

(42 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third-party centralized randomization also adds up to allocation concealment being secured until the end of the study. Thus, the usage of sequentially numbered opaque sealed envelopes should be rendered obsolete, especially since we demonstrated it is widely used or described incompletely [10]. If systematic review authors do not find information about in research reports regarding specific methods for randomization and allocation concealment, they should be careful to avoid making erroneous RoB judgment.…”

Section: Discussionmentioning

confidence: 99%

“…Cochrane reviews published between July 2015 and June 2016 in the Cochrane Database of Systematic Reviews (CDSR) were analyzed. This was a convenient one-year sample based on our previous studies [7,8,10,16], from the period of 4 years after introduction of the 2011 RoB tool. The reviews that have included RCTs only, or both RCTs and non-randomized studies were found eligible.…”

Section: Inclusion and Exclusion Criteriamentioning

confidence: 99%

“…The source for our assessments was the accompanying comment from the RoB table and description of the intervention provided by the Cochrane authors. For the RoB domain for random sequence generation and allocation concealment accompanying comments were categorized to bring judgment as described in our previous studies [8,10]. For the blinding domains, we needed to determine which subject was blinded and whether the outcome(s) were susceptible to lack of blinding [7,16].…”

Section: Assessment Of Adequacy For Four Domains Of Risk Of Bias Toolmentioning

confidence: 99%

“…However, RoB assessments made by authors of published systematic reviews should not be taken at the face value, as we have shown in multiple studies that RoB assessments in many Cochrane reviews were inadequate and inconsistent [7][8][9][10][11][12][13][14]. Due to the specificities of surgical trials, we hypothesized that assessments of surgical trials may be more accurate and more consistent, compared to RCTs of nonsurgical interventions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Barčot

Borić

Došenović

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background Bias in randomized controlled trials (RCTs) can lead to underestimation or overestimation of the true effects of interventions. Surgical RCTs may suffer from the risk of bias (RoB) that is avoidable in trials of other interventions, and vice versa. We aimed to compare the adequacy of RoB assessments in surgical versus non-surgical RCTs included in Cochrane reviews and to assess the most common differences in those RoB assessments. Due to specificities of surgical trials, i.e. difficulties associated with blinding of surgical interventions, we hypothesized that assessments of surgical trials may be more adequate, compared to RCTs of non-surgical interventions. Methods This was a methodological study, analyzing methods of published Cochrane systematic reviews. Data were extracted from RoB tables in Cochrane reviews (judgments and accompanying explanatory comment) for the following four RoB domains used in the 2011 Cochrane RoB tool: randomization, allocation concealment, blinding of participants and personnel, and blinding of outcome assessors. We defined adequate assessments as those that were in line with instructions from the Cochrane Handbook for Systematic Reviews of Interventions. The prevalence of adequate assessments was compared in surgical versus non-surgical trials. The most common differences in both groups of reviews were presented. Results In 729 analyzed Cochrane reviews, there were 10,537 included trials. The prevalence of adequate RoB judgments made by Cochrane authors ranged from 87.9, 95%CI (87.3 to 88.6%) for randomization to 70.7, 95%CI (69.8 to 71.5%) for blinding of participants and personnel. For all analyzed RoB domains, the prevalence of adequate RoB domains was higher in surgical trials than in non-surgical trials. For two RoB domains assessing blinding, this difference between surgical and non-surgical trials was statistically significant (P < 0.001), while the difference was not significant for the RoB domain regarding randomization (P = 0.124) and allocation concealment (P = 0.039, β < 0.8). Conclusions RoB judgments were more in line with instructions from the Cochrane Handbook when Cochrane reviews assessed surgical trials, compared to those that analyzed non-surgical interventions. However, further steps are warranted to scrutinize RoB assessment in trials of both surgical and non-surgical interventions.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Inclusion and Exclusion Criteriamentioning

confidence: 99%

Section: Assessment Of Adequacy For Four Domains Of Risk Of Bias Toolmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Barčot

Borić

Došenović

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

show abstract

“…Cochrane risk of bias (RoB) tool, with seven dimensions, including sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting and other potential threats to study validity will be used. We will assess RoB in line with instructions from the Cochrane Handbook for Systematic Reviews of Interventions22 to avoid mistakes review authors frequently make when conducting RoB assessments with this tool 23–27. For each study, we will create a RoB domain, which will have a RoB judgement for each domain (high, low or unclear risk) and an accompanying supporting comment to explain the RoB judgement.…”

Section: Methods and Analysismentioning

confidence: 99%

Efficacy and safety of modified-release paracetamol for acute and chronic pain: a systematic review protocol

2019

View full text Add to dashboard Cite

IntroductionParacetamol (acetaminophen) is widely used for management of mild-to-moderate pain and reduction of fever. It is available as immediate release (IR) and modified-release (MR) formulation. In 2017, European Medicines Agency recommended a suspension of marketing of MR paracetamol in the European Union. Benefit-risk balance of these products has been assessed as negative as data showed that existing procedures for overdose management may not be efficient. Since MR paracetamol is still available in other countries (Australia and USA) and there is no available systematic review (SR) of efficacy and safety of MR paracetamol in the literature, we have decided to perform one to evaluate available data from randomised clinical trials (RCTs).Methods and analysisUsing predefined search criteria, we will search EMBASE, MEDLINE, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and WHO International Clinical Trials Registry Platform to identify RCTs evaluating efficacy and safety of MR paracetamol alone in any dose or duration for any pain. Participants are defined as adults and adolescents (over 12 years). Primary efficacy outcomes will be pain intensity, pain relief and sleep. Primary safety outcomes will be the number of patients experiencing any (serious) adverse event, the number of patients withdrawn due to adverse events and the number of patients with gastrointestinal and hepatic adverse events. Data analysis will be subdivided based on different clinical syndromes. Meta-analysis will be conducted if possible. Cochrane risk of bias (RoB) tool with seven dimensions will be used to assess RoB of individual studies.Ethics and disseminationThis SR will include only data collected from trial reports; therefore, an ethical approval will not be sought. We will publish the protocol and our findings in peer-reviewed journals.PROSPERO registration numberNCT42018115769.

show abstract

Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: A randomized controlled trial

Barčot

Ivanda

Buljan

et al. 2021

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

This randomized controlled trial (RCT) aimed to test the efficacy of enhanced access to Cochrane Handbook (Handbook) recommendations for judging the 2011 Cochrane risk of bias (RoB) domains for improving the adequacy of RoB judgments. Parallel‐group RCT with a 1:1 allocation ratio (N = 2271 per group) was conducted. Eligible participants were corresponding authors of all published Cochrane reviews and protocols. After allocation by a random number generator, participants received 20 scenarios for assessing RoB. The intervention group was shown tables from the Handbook with instructions for assessing 2011 RoB tool together with scenarios they were supposed to assess—enhanced access to the Handbook. The control group was shown only a general link to the Handbook. The primary outcome was the proportion of participants that made an adequate judgment of RoB scenarios for analyzed domains. There were 240 responses out of 2020 delivered e‐mail invitations in the intervention and 197/2254 in the control group. Only five participants from the intervention group judged RoB adequately in all the 20 scenarios and no one in the control group. The proportion of participants who adequately assessed all the scenarios within a domain was significantly higher in the intervention than in the control group. The frequency of adequate RoB judgments was 7.1% (95% CI: 5.0–9.3%, p < 0.001) higher in the intervention group (76.2%) than in the control group (69.0%). The enhanced access yields more adequate RoB assessments and could be incorporated in software supporting the RoB tool.

show abstract

In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance

Cited by 50 publications

References 12 publications

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Efficacy and safety of modified-release paracetamol for acute and chronic pain: a systematic review protocol

Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: A randomized controlled trial

Contact Info

Product

Resources

About