An appraisal of emergency medicine clinical practice guidelines: Do we agree?

Abstract

Background: Clinical practice guidelines (CPGs) have been published by the American College of Emergency Physicians (ACEP) since 1990 to advance evidence-based emergency care. ACEP clinical policies have drawn anecdotal criticism for bias, yet the overall quality of these guidelines has not previously been quantified. We sought to examine ACEP clinical policies using a recognized, validated appraisal instrument: Appraisal of Guideline for Research & Evaluation (AGREE II). Methods: Systematic assessment of current ACEP clinical policies using the AGREE II instrument, which contains 23 appraisal items (scored on a 1-7 scale) in six domains and two overall assessments. Each policy was independently appraised by five trained appraisers. Primary outcomes were AGREE II ratings for each item, domain, and "Overall Assessment," and scores were reported as standardized percentages from all five appraisers. Secondary analyses examined associations between AGREE II ratings and policy publication date, strength of underlying evidence, and strength of recommendations. Additional analysis examined relationships between domain and "Overall Assessment" ratings. Results: Twenty guidelines published from April 2007 to November 2017 were included. Of the six domains, "Scope and Purpose" scored highest (mean 90%) and "Applicability" scored lowest (mean 35%). The four remaining domains ("Stakeholder Involvement," "Rigor of Development,"

The structure of the quality of clinical practice guidelines with the items and overall assessment in AGREE II: a regression analysis

“…Previous studies regarding the quality of CPGs were limited to specific health topics or regions [21–39] and systematic reviews using these studies [40–42]. Regarding the relationship between quality and application of CPGs, O’Sullivan et al clarified that high “scores in some domains of AGREE II tool were significantly associated with reductions in nonadherent testing” [32].…”

“…Therefore, AGREE II items and domains can affect the overall assessment. Although several studies have revealed the correlation between domain scores and the overall assessment, they did not adjust the influence between domains [30, 39, 40]. Adjusting such influence, Hoffman-Eßer et al demonstrated the influence of domains on the overall assessment [42].…”

Quality Evaluation of Clinical Practice Guidelines for Comprehensive Patient Assessment in Emergency Care: A National Cross-sectional Survey

“…Thus, the CPG implementation process in emergency care settings is de cient. This lack may be explained by CPG development and implementation processes being separate activities (48), or it may exist because CPG developers are not involved in everyday emergency clinical work. In summary, our result con rms that existing evidenced-based CPGs are not in use in Swedish emergency care and that the shortage of evidence-based CPGs may result in patients not being provided the best care and treatment.…”

Navigating uncertainty with GRACE: Society for Academic Emergency Medicine’s guidelines for reasonable and appropriate care in the emergency department

“…If not GRADE, then what framework should GRACE use? EM stakeholders’ evaluation of 20 clinical policies using the Appraisal of Guideline for Research and Evaluation (AGREE II) instrument noted significant problems with applicability and issues with stakeholder involvement and clarity of presentation of current guidelines 33 . Furthermore, a similar analysis of the levels of evidence supporting ACEP policy recommendations found that the majority of these were supported by lower classes of evidence and primarily expert opinion based 34 .…”