What is “quality of evidence” and why is it important to clinicians?

Abstract

Guideline developers use a bewildering variety of systems to rate the quality of the evidence underlying their recommendations. Some are facile, some confused, and others sophisticated but complexIn 2004 the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group presented its initial proposal for patient management. In this second of a series of five articles focusing on the GRADE approach to developing and presenting recommendations we show how GRADE has built on previous systems to create a highly structured, transparent, and informative system for rating quality of evidence.A guideline's formulation should include a clear question Any question addressing clinical management has four components: patients, an intervention, a comparison, and the outcomes of interest.2 For example, consider the following: in patients with pancreatic carcinoma undergoing surgery, what is the impact of a modified resection that preserves the pylorus compared with a standard wide tumour resection-variations of the Whipple procedure-on short term and long term mortality, blood transfusions, bile leaks, hospital stay, and problems with gastric emptying?Guideline developers should address the importance of their outcomes GRADE challenges guideline developers to specify all outcomes that are of importance to patients as they begin the guideline development process, and to differentiate the critical outcomes from the important but not critical ones.3 Figure 1 presents a hierarchy of patient important outcomes regarding the impact of phosphate lowering drugs in patients with renal failure. GRADE suggests a nine point scale to judge importance. The upper end of the scale, 7 to 9, identifies outcomes of critical importance for decision making. Ratings of 4 to 6 represent outcomes that are important but not critical. Ratings of 1 to 3 are items of limited importance. Guideline panels should strive for the sort of explicit approach that this example represents.Judging the quality of evidence requires consideration of the context GRADE provides a definition for the quality of evidence in the context of making recommendations. The quality of evidence reflects the extent to which confidence in an estimate of the effect is adequate to support recommendations. This definition has two important implications. Firstly, guideline panels must make judgments about the quality of evidence relative to the specific context in which they are using the evidence. Secondly, because systematic reviews do not-or at least should not-make recommendations, they require a different definition. For systematic reviews, the quality of evidence reflects the extent of confidence that an estimate of effect is correct.Study design is important in determining the quality of evidence As with early systems of grading the quality of evidence, 4 GRADE's approach begins with the study design. For recommendations addressing alternative management strategies-as opposed to issues of establishing prognosis or the accuracy of diagnostic testsrandomise...

Intensive care diaries to promote recovery for patients and families after critical illness: A Cochrane Systematic Review

“…The quality of the evidence contained in the review has been assessed using the GRADE approach [53]. While publication bias, indirectness and inconsistency were not established, the methodologic quality and precision of the effect estimates was low to very low.…”

A Framework for Crafting Clinical Practice Guidelines that are Relevant to the Care and Management of People with Multimorbidity

“…18 We organized our work by the generic CPG development steps extracted from the review of leading, collaborative CPG initiatives. [19][20][21][22][23][24][25][26][27] Next, we iteratively drafted and refined a list of issues and existing approaches relevant for addressing multimorbidity in each step of CPG development, based on review of key literature and investigators' experience. An expert panel (see acknowledgements, also co-authors) with clinical and methods expertise in CPG development, methods of evidence synthesis, epidemiology, and multimorbidity provided feedback on the list of issues identified by CB, KU, BL, and were asked to identify examples of how existing CPGs currently address or do not address multimorbidity.…”

Scandinavian SSAI clinical practice guideline on choice of first‐line vasopressor for patients with acute circulatory failure

“…GRADE classifies recommendations as ‘strong’ when virtually all informed patients would choose the recommended management strategy. ‘Weak’ recommendations apply when fully informed patients would choose different management strategies, and reflects a close call between benefits and harms, uncertainty regarding treatment effects, questionable cost‐effectiveness, or variability in values and preferences 5, 9. The group agreed upon all the recommendations in this guideline.…”