Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis

Abstract

Background The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement. Methods We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015. Results In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8–32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6–16%. Conclusions NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.

Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement

“…Although few published evaluations exist, evidence suggests that the endorsement of PRISMA by journals is associated with more complete reporting of systematic reviews than in journals not endorsing the reporting guideline. 50 51 52 We will actively contact journal editors to seek their endorsement of PRIOR and recommend that their authors use the checklist to inform their reporting, require a completed PRIOR checklist and flow diagram for all submitted overviews of reviews, and encourage use by peer reviewers.…”

“…Our dissemination plan will include housing the checklist on the EQUATOR network website (https://www.equator-network.org/ ) ; launching a PRIOR website or the addition of a tab to the PRISMA website, which receives high traffic; developing infographics or tip sheets; preparing a slide deck for those who wish to incorporate PRIOR into their teaching; and sharing the PRIOR statement widely via social media channels (eg, Twitter), email listservs, and in relevant newsletters. Although few published evaluations exist, evidence suggests that the endorsement of PRISMA by journals is associated with more complete reporting of systematic reviews than in journals not endorsing the reporting guideline 505152. We will actively contact journal editors to seek their endorsement of PRIOR and recommend that their authors use the checklist to inform their reporting, require a completed PRIOR checklist and flow diagram for all submitted overviews of reviews, and encourage use by peer reviewers.…”

Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: a meta-research study

“…mention of a review protocol) remained infrequently reported (13). Similarly, improvements in the reporting of network meta-analysis (NMA) were observed after the introduction of the PRISMA-NMA extension (16); however, the authors concluded that the improvements could not be attributed solely to the availability of this extension. The extent to which adoption of reporting guidelines influences completeness of reporting remains unclear.…”

Reporting guideline for overviews of reviews of healthcare interventions: The Preferred Reporting Items for Overviews of Reviews (PRIOR) statement

“…Our dissemination plan will include: housing the checklist on the EQUATOR network website (https://www.equator-network.org/); launching a PRIOR website or the addition of a tab to the PRISMA website, which receives high traffic; developing infographics and/or tip sheets; preparing a slide deck for those who wish to incorporate PRIOR into their teaching; and sharing the PRIOR statement widely via social media channels (e.g., Twitter pages), email listservs, and in relevant newsletters. Though few published evaluations exist, there is evidence to suggest that the endorsement of PRISMA by journals is associated with more complete reporting of systematic reviews, compared to journals not endorsing the reporting guideline (40)(41)(42). We will actively contact journal editors to seek their endorsement of PRIOR and recommend that their authors use the checklist to inform their reporting, require a completed PRIOR checklist and flow diagram for all submitted overviews of reviews, and encourage use by peer-reviewers.…”