Reporting of Cochrane systematic review protocols with network meta‐analyses—A scoping review

Abstract

Publishing systematic review protocols is a fundamental part of systematic reviews to ensure transparency and reproducibility. In this scoping review, we aimed to evaluate reporting of Cochrane systematic review protocols with network meta‐analyses (NMA). We searched all Cochrane NMA protocols published in 2018 and 2019, and assessed the characteristics and reporting of methodologies relevant to NMA. We reported frequencies for each reporting item. Forty‐five protocols were assessed, including two for overviews and 43 for intervention reviews. Thirty‐three (73%) were labelled as NMA protocols in the title. Forty‐two (95%) justified the need of an NMA and 40 (89%) used appropriate search strategies to identify potential eligible studies. About half (24, 53%) considered the transitivity assumption when reporting inclusion criteria and 35 (78%) specified potential effect modifiers. Forty‐three (96%) reported statistical software for NMA, 25 (56%) reported NMA model choice, 32 (71%) reported framework choice and 32 (71%) reported assumption about heterogeneity variances. Protocols varied in whether they reported methods for relative ranking (35, 78%), statistical inconsistency (40, 89%), reporting bias (44, 98%) and sources of heterogeneity (39, 87%). In conclusion, Cochrane NMA protocols reported multiple NMA‐specific items well, but could be further improved, especially regarding transitivity assumptions. Our recommendations for NMA protocol reporting based on this scoping review could assist authors, reviewers, and editors to improve NMA protocols.

Laser therapy decreases oral leukoplakia recurrence and boosts patient comfort: a network meta-analysis and systematic review

“…The inconsistency factor (IF) and its 95% CI were derived using a z-test to assess potential discrepancies between direct and indirect comparisons within the network. An IF value close to zero and a 95% CI containing zero indicate consistency between direct and indirect estimates [ 23 , 24 ]. Funnel plots were constructed to assess the presence of small study effects or publication bias [ 25 ].…”

Effectiveness of Rehabilitation for Osteoarthritis of the Knee: A Scoping Review of Network Meta-Analyses

“…Frameworks of the analysis: In the included reviews, the Bayesian (n=4 [50%]) and frequentist (n=4 [50%]) frameworks were used for analyses. However, previous reviews of a larger number of NMAs revealed that the Bayesian framework seemed more popular than the frequentist framework (usage: 66.7% vs. 32.2% in one study [ 12 ] and 51% vs. 20% in another study [ 21 ]).…”

“…Software for statistical analysis: The NMAs included in this review were analyzed using WinBUGS, R, and Stata; however, OpenBUGS was a popular choice as well [ 21 ]. Practitioners need to access these four software and work closely with biostatisticians [ 23 ].…”

“…Method for evaluating transitivity: Two of the eight NMAs included in this review confirmed the transitivity assumption by assessing potential effect modifiers [ 29 , 30 ]. In a recent review of 45 Cochrane NMA protocols [ 21 ], transitivity assumptions were considered in the reporting of the inclusion criteria in approximately half of the cases; furthermore, possible effect modifiers were specified in 78% of the cases. While planning future NMAs, a more careful evaluation of the effect modifiers and transitivity assumptions should be considered [ 47 ].…”

Regional block techniques for pain management after video-assisted thoracoscopic surgery: a covariate-adjusted Bayesian network meta-analysis.

“…However, in fact, we have another important issue that should not be overlooked: network meta-analysis tends to be highly heterogeneous. Network meta-regression is one of the main ways to address heterogeneity [ 7 ], and covariates should be discussed in the analysis. All the net meta-analyses related to this topic have lacked adjustment for covariates.…”