Methodological Quality of Systematic Reviews and Meta‐analyses Published in High‐Impact Otolaryngology Journals

Martinez-Monedero, Rodrigo; Danielian, Arman; Angajala, Varun; Dinalo, Jennifer; Kezirian, Eric J.

doi:10.1177/0194599820924621

Cited by 14 publications

(8 citation statements)

References 244 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…53 Moreover, Martinez-Mondero et al concluded that "over 50% of these SRs and MAs had weaknesses in at least 3 of the 16 items in the AMSTAR-2." 54 Taken together, our findings are consistent with the existing literature across multiple disease processes within otolaryngology and other medical specialties. This issue can potentially be addressed by having journal editors require submission of the AMSTAR-2 checklist alongside the SR.…”

Section: Discussionsupporting

confidence: 90%

Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer

et al. 2022

View full text Add to dashboard Cite

Objective Clinical practice guidelines (CPGs) are essential to clinical decision‐making as their recommendations are supported by published literature. Systematic reviews are considered the highest quality of evidence used to underpin these guidelines. However, research to support these recommendations may lack compliance to quality reporting among systematic reviews (SRs). Here, we aim to evaluate the quality of SRs underpinning CPG recommendations for the management of head and neck cancer (HNC). Study Design Retrospective cross‐sectional analysis. Methods Using PubMed, we searched for CPGs pertinent to the management of head and neck cancer published between January 2017 and May 2021. Relevant guidelines were analyzed for all SRs. Cited SRs in CPGs were evaluated using A Measurement Tool to Assess Systematic Reviews‐2 (AMSTAR‐2) and Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) instruments. Study characteristics were extracted in a masked triplicate fashion. Results A total of 16 CPGs and 142 unique SRs were included in our study. PRISMA completion ranged from 67.15% to 87.65% across CPGs with a mean of 76.41% (SD = 16.9). AMSTAR‐2 completion ranged from 34.38% to 84.38% across CPGs with a mean of 67.55% (SD = 20.9) among all SRs. The lowest rated items included funding sources and publication bias. A higher score was achieved in SR done by Cochrane group and it was only 2.11% (3/142) of all SR's quoted in CPG. Conclusion Adherence to AMSTAR‐2 and PRISMA items exhibits a variation among SRs cited in CPGs for the management of HNC. The mature underpinning CPG recommendation of SRs cited as supportive evidence could be enhanced in reporting quality. Level of Evidence NA Laryngoscope, 132:1976–1983, 2022

show abstract

Section: Discussionsupporting

confidence: 90%

Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer

et al. 2022

View full text Add to dashboard Cite

show abstract

“…To aid in evaluating the quality of meta-analyses, a number of critical appraisal tools are now available (eg, A Measurement Tool to Assess Systematic Reviews 2, AMSTAR 2) 12. Applied in a variety of areas of biomedical research, these tools have provided empirical evidence for areas of improvement in future meta-analyses, yet no reviews exist within the context of CPSP 13–15…”

Section: Introductionmentioning

confidence: 99%

Quality of meta-analyses of non-opioid, pharmacological, perioperative interventions for chronic postsurgical pain: a systematic review

McGregor

Warner

Linde

et al. 2022

Reg Anesth Pain Med

View full text Add to dashboard Cite

BackgroundIn an attempt to aggregate observations from clinical trials, several meta-analyses have been published examining the effectiveness of systemic, non-opioid, pharmacological interventions to reduce the incidence of chronic postsurgical pain.ObjectiveTo inform the design and reporting of future studies, the purpose of our study was to examine the quality of these meta-analyses.Evidence reviewWe conducted an electronic literature search in Embase, MEDLINE, and the Cochrane Database of Systematic Reviews. Published meta-analyses, from the years 2010 to 2020, examining the effect of perioperative, systemic, non-opioid pharmacological treatments on the incidence of chronic postsurgical pain in adult patients were identified. Data extraction focused on methodological details. Meta-analysis quality was assessed using the A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2) critical appraisal tool.FindingsOur search yielded 17 published studies conducting 58 meta-analyses for gabapentinoids (gabapentin and pregabalin), ketamine, lidocaine, non-steroidal anti-inflammatory drugs, and mexiletine. According to AMSTAR 2, 88.2% of studies (or 15/17) were low or critically low in quality. The most common critical element missing was an analysis of publication bias. Trends indicated an improvement in quality over time and association with journal impact factor.ConclusionsWith few individual trials adequately powered to detect treatment effects, meta-analyses play a crucial role in informing the perioperative management of chronic postsurgical pain. In light of this inherent value and despite a number of attempts, high-quality meta-analyses are still needed.PROSPERO registration numberCRD42021230941.

show abstract

“…Issues involved in meeting the standards for seven review components (identified in bold in Table 4.1) are addressed in detail. These were chosen for elaboration for one (or both) of two reasons: 1) The component has been identified as potentially problematic for systematic review authors based on consistent reports of their frequent AMSTAR-2 or ROBIS deficiencies (Gagnier & Kellam, 2013;Kolaski et al, 2021;Martinez-Monedero et al, 2020;Pussegoda et al, 2017;Riado Minguez et al, 2017;Tsoi et al, 2020); and/or 2) the review component is judged by standards of an AMSTAR-2 "critical" domain. These have the greatest implications for how a systematic review will be appraised: if standards for any one of these critical domains are not met, the review is rated as having "critically low confidence.…”

Section: Acronym Year Linkmentioning

confidence: 99%

Guidance to best tools and practices for systematic reviews

Kolaski

Logan

Ioannidis

2023

British J Pharmacology

View full text Add to dashboard Cite

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well‐documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in‐depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

show abstract

Methodological Quality of Systematic Reviews and Meta‐analyses Published in High‐Impact Otolaryngology Journals

Cited by 14 publications

References 244 publications

Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer

Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer

Quality of meta-analyses of non-opioid, pharmacological, perioperative interventions for chronic postsurgical pain: a systematic review

Guidance to best tools and practices for systematic reviews

Contact Info

Product

Resources

About