Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain

Minguez, Daniel Riado; Kowalski, Martin; Odena, Marta Vallve; Pontzen, Daniel Longin; Kadić, Antonia Jeličić; Jerić, Milka; Došenović, Svjetlana; Jakus, Dora; Vrdoljak, Marija; Peričić, Tina Poklepović; Sapunar, Damir; Puljak, Livia

doi:10.1213/ane.0000000000002227

Cited by 38 publications

(31 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, these may be some common methodological shortcomings of any types of SRs. In SRs, assessing the risk of bias of primary studies and investigating publication bias are of great importance as these will gauge the validity of meta-analytic results, affect the interpretation of results, and limit our ability to draw conclusions [43,44]. Unfortunately, only 28.6% of IPD-NMAs used a satisfactory technique for assessing the RoB in individual studies included, and only 19% of IPD-NMAs assessed the potential impact of RoB on the results of the meta-analyses.…”

Section: Findings and Interpretationsmentioning

confidence: 99%

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

Gao

Shi

et al. 2020

BMC Med

View full text Add to dashboard Cite

Background: Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs. Methods: We searched four bibliographic databases to identify published IPD-NMAs. The methodological quality was assessed using AMSTAR-2 and reporting quality assessed based on PRISMA-IPD and PRISMA-NMA. We performed stratified analyses and correlation analyses to explore the factors that might affect quality. Results: We identified 21 IPD-NMAs. Only 23.8% of the included IPD-NMAs reported statistical techniques used for missing participant data, 42.9% assessed the consistency, and none assessed the transitivity. None of the included IPD-NMAs reported sources of funding for trials included, only 9.5% stated pre-registration of protocols, and 28.6% assessed the risk of bias in individual studies. For reporting quality, compliance rates were lower than 50.0% for more than half of the items. Less than 15.0% of the IPD-NMAs reported data integrity, presented the network geometry, or clarified risk of bias across studies. IPD-NMAs with statistical or epidemiological authors often better assessed the inconsistency (P = 0.017). IPD-NMAs with a priori protocol were associated with higher reporting quality in terms of search (P = 0.046), data collection process (P = 0.031), and syntheses of results (P = 0.006). Conclusions: The reporting of statistical methods and compliance rates of methodological and reporting items of IPD-NMAs were suboptimal. Authors of future IPD-NMAs should address the identified flaws and strictly adhere to methodological and reporting guidelines.

show abstract

Section: Findings and Interpretationsmentioning

confidence: 99%

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

Gao

Shi

et al. 2020

BMC Med

View full text Add to dashboard Cite

show abstract

“…However, most non-Cochrane reviews only provided a list of included studies. If we rely on inappropriate reasons to exclude studies, it would introduce bias, which would in uence the transparency, validity and reported quality of our results 51,52 . Hence, it was obvious that Cochrane reviews showed better methodological quality than non-Cochrane review 46,47 .…”

Section: Discussionmentioning

confidence: 99%

Methodological Quality Assessment of Systematic Review or Meta-Analysis Using AMSTAR-2: the Long-Term Effectiveness or Efficacy of Opioids for Chronic Non-Cancer Pain

Deng

Jiang³

et al. 2019

Preprint

View full text Add to dashboard Cite

Background：Systematic review or meta-analysis, the strong study design of high quality evidence, give inconsistent conclusion of long-term effectiveness or efficacy of opioids for chronic non-cancer pain. We appraised the methodological quality of systematic reviews or meta-analyses. Methods: We found the relevant systematic reviews or meta-analyses by searching Medline, EMBASE, the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the International prospective register of systematic reviews, Psyc ARTICLES/OVID, the Chinese Bio-Medical Literature Database, the China National Knowledge Infrastructure, and the Wan Fang Data and VIP Database on March 1st, 2019. The methodological quality was assessed by A Measurement Tool to Assess Systematic Reviews-2(AMSTAR-2). Spearman correlation analysis and non-parametric tests were used to assess the association between quality and factors. Results: Twenty-one systematic reviews or meta-analyses were included in our study. One has no individual study. In terms of methodological quality, twelve reviews were critically low in overall confidence, four reviews were low, two reviews were moderate, two reviews were high. When referring to the systematic reviews or meta-analyses of relatively better methodological quality with more credible results and conclusions, the effectiveness or efficacy of opioids was small to questionable. Cochrane reviews performed better than non-Cochrane reviews in establishing prior protocol (100% vs 17%, P<0.05), providing an excluded studies list (100% vs 50%, P<0.05) and taking risk of bias into account when interpreting the results of the review (100% vs 75%, P<0.05). There was a strong correlation (ρ=0.526, P<0.05) between the impact factor of systematic reviews or meta-analyses in published journals and methodological quality. Conclusion The methodological quality of the included systematic reviews or meta-analyses is far from satisfactory and needs improvement, especially in establishing prior protocol and justifying significant deviations from the protocol, providing an excluded primary studies list, reporting the funding information of primary studies, and assessing the potential impact of risk of bias on individual studies.

show abstract

“…In the methodological study by Thabut et al, journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine "high impact" [77]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [78].…”

Section: Some Frequently Asked Questions About Methodological Studiesmentioning

confidence: 99%

A tutorial on methodological studies: the what, when, how and why

Mbuagbaw

Lawson

Puljak

et al. 2020

BMC Med Res Methodol

Self Cite

View full text Add to dashboard Cite

Background Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste. Main body We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies? Conclusion Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

show abstract

Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain

Abstract: Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.

Cited by 38 publications

References 22 publications

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

Methodological Quality Assessment of Systematic Review or Meta-Analysis Using AMSTAR-2: the Long-Term Effectiveness or Efficacy of Opioids for Chronic Non-Cancer Pain

A tutorial on methodological studies: the what, when, how and why

Contact Info

Product

Resources

About