An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews

Pollock, Alex; Farmer, Sybil E.; Brady, Marian; Langhorne, Peter; Mead, Gillian; Mehrholz, Jan; Wijck, Frederike van; Wiffen, Philip J

doi:10.1016/j.jclinepi.2015.08.013

Cited by 125 publications

(135 citation statements)

References 14 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…There is an absence of guidance on how to apply GRADE within an overview. Authors using GRADE faced challenges relating to the number of comparisons, and subtle differences between comparisons, which created issues in terms of workload and in relation to achievement consistency.Summary of findings from Ballard [3]:Pollock [19] identified challenges in consistent application of the GRADE approach to large volumes of evidence synthesised within overviews [37], and proposedla more algorithmic approach to judging quality of evidence within reviews [10]. There remains debate about the validity of this proposed approach [11, 12], but Pollock [19] argues that approach does arguably facilitate transparency and consistency when faced with judging the quality of evidence of many similar (but not identical) comparisons included within reviews.• “emerging debate related to (iii) evaluating the quality and reporting of included research” (“quality of the body of evidence across included systematic reviews”)• GRADE has been described as an approach for assessing the quality of the body of evidence accrsoss systematic reviews, but there is currently a lack of guidance to ensure appropriate use and interpretation of GRADE when applied in this wayMcClurg [21] repeated this method and developed algorithm using the method recommended by Pollock [19], but involving a wider group of people in the decision making, including statisticians and clinicians.Judgement of complementarityAgreementEstcourt [20] used GRADE levels of evidence from within included reviews.…”

Section: Resultsmentioning

confidence: 99%

Selecting and implementing overview methods: implications from five exemplar overviews

et al. 2017

Self Cite

View full text Add to dashboard Cite

BackgroundOverviews of systematic reviews are an increasingly popular method of evidence synthesis; there is a lack of clear guidance for completing overviews and a number of methodological challenges. At the UK Cochrane Symposium 2016, methodological challenges of five overviews were explored. Using data from these five overviews, practical implications to support methodological decision making of authors writing protocols for future overviews are proposed.MethodsMethods, and their justification, from the five exemplar overviews were tabulated and compared with areas of debate identified within current literature. Key methodological challenges and implications for development of overview protocols were generated and synthesised into a list, discussed and refined until there was consensus.ResultsMethodological features of three Cochrane overviews, one overview of diagnostic test accuracy and one mixed methods overview have been summarised. Methods of selection of reviews and data extraction were similar. Either the AMSTAR or ROBIS tool was used to assess quality of included reviews. The GRADE approach was most commonly used to assess quality of evidence within the reviews.Eight key methodological challenges were identified from the exemplar overviews. There was good agreement between our findings and emerging areas of debate within a recent published synthesis. Implications for development of protocols for future overviews were identified.ConclusionsOverviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-017-0534-3) contains supplementary material, which is available to authorized users.

show abstract

Section: Resultsmentioning

confidence: 99%

Selecting and implementing overview methods: implications from five exemplar overviews

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, AMSTAR-2 proposes a four-level scheme (high, moderate, low, and critically low) for appraisers to rate the overall confidence in the results of a systematic review, and each item was evaluated using three evaluation options, "yes," "partial yes," and "no." e Grading of Recommendation, Assessment, Development, and Evaluation (GRADE) [12] was used to assess the quality of evidence by two reviewers (JK-H and M-S) independently. e following criteria were taken into account: risk of bias (that is study limitations), inconsistencies, indirectness, inaccuracy, and publication bias [13].…”

Section: Quality Assessment Two Reviewers (Jk-h and M-s)mentioning

confidence: 99%

Acupuncture for the Treatment of Tension‐Type Headache: An Overview of Systematic Reviews

Huang

Shen

Qin

et al. 2020

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

Objectives. Because current evidence regarding the effectiveness of acupuncture for a tension-type headache (TTH) is controversial, we evaluated the reliability of the methodological quality and outcome measures of systematic reviews/metaanalyses (SRs/MAs) on acupuncture for TTH. Methods. We conducted a comprehensive literature search for SRs/MAs in major databases from the database's inception to September 2019. e Methodological Quality of Systematic Reviews 2 (AMSTAR-2) and the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) assessments were used to assess the methodological quality of the included reviews and the quality of evidence, respectively. Results. Eight reviews were included in the analysis. e AMSTAR-2 assessment results showed that the methodological quality of all included reviews was critically low. irty-six outcome measures were included in these reviews. e GRADE results showed that 25 (25/36, 69.4%) outcomes provided low-or very low-quality evidence, four (4/36, 11.1%) provided moderate-quality evidence, and seven (7/36, 19.4%) provided high-quality evidence. Descriptive analysis results showed that acupuncture treatment for TTH reduced headache frequency and severity. Conclusions. Acupuncture appears to be an effective treatment modality for TTH, but the credibility of the results is limited owing to the generally low methodological quality and evidence quality in the included SRs/MAs.

show abstract

“…Empirical evidence is lacking on the optimal tool for assessing risk of bias or methodological quality of included systematic reviews, and how these tools might best be applied in overviews of reviews [30,31]. Guidance remains limited on how to extract and use appraisals of the quality of primary studies within the included systematic reviews and how to adapt GRADE methodology to overviews of reviews [7,23]. The challenges that overview authors reportedly face are often related to the steps where guidance is inadequate or con icting.…”

Section: Discussionmentioning

confidence: 99%

“…On 7 March 2019 we conducted an iterative reference tracking ('snowballing') search [21,22]. We used 46 target articles, including all published articles and abstracts cited in the 2016 scoping review [23], as well as other recent relevant articles known to the research team. For each target article, we searched for 'citing' references in Google Scholar and Scopus and for 'similar articles' in PubMed from 1 January 2014 to present.…”

Section: Searchesmentioning

confidence: 99%

Guidance for Overviews of Reviews Continues to Accumulate, But Important Challenges Remain: A Scoping Review

Gates

Guitard

et al. 2020

Preprint

View full text Add to dashboard Cite

Background. Overviews of reviews (overviews) provide an invaluable resource for healthcare decision-making by combining large volumes of systematic review (SR) data into a single synthesis. The production of high-quality overviews hinges on the availability of practical evidence-based guidance for conduct and reporting.Objectives. Within the broad purpose of informing the development of a reporting guideline for overviews, we aimed to provide an up-to-date map of existing guidance related to the conduct of overviews, and to identify common challenges that authors face when undertaking overviews.Methods. We updated a scoping review published in 2016 using the search methods that had produced the highest yield: ongoing reference tracking (2014 to March 2020 in Pubmed, Scopus, and Google Scholar), hand-searching conference proceedings and websites, and contacting authors of published overviews. Using a qualitative meta-summary approach, one reviewer extracted, organized, and summarized the guidance and challenges presented within the included documents. A second reviewer verified the data and synthesis.Results. We located 28 new guidance documents, for a total of 77 documents produced by 34 research groups. The new guidance helps to resolve some earlier identified challenges in the production of overviews. Important developments include strengthened guidance on handling primary study overlap at the study selection and analysis stages. Despite marked progress, several areas continue to be hampered by inconsistent or lacking guidance. There is ongoing debate about whether, when, and how supplemental primary studies should be included in overviews. Guidance remains scant on how to extract and use appraisals of quality of the primary studies within the included SRs and how to adapt GRADE methodology to overviews. The challenges that overview authors face are often related to the above-described steps in the process where evidence-based guidance is lacking or conflicting.Conclusion. The rising popularity of overviews has been accompanied by a steady accumulation of new, and sometimes conflicting, guidance. While recent guidance has helped to address some of the challenges that overview authors face, areas of uncertainty remain. Practical tools supported by empirical evidence are needed to assist authors with the many methodological decision points that are encountered in the production of overviews.

show abstract

An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews

Cited by 125 publications

References 14 publications

Selecting and implementing overview methods: implications from five exemplar overviews

Selecting and implementing overview methods: implications from five exemplar overviews

Acupuncture for the Treatment of Tension‐Type Headache: An Overview of Systematic Reviews

Guidance for Overviews of Reviews Continues to Accumulate, But Important Challenges Remain: A Scoping Review

Contact Info

Product

Resources

About