World Health Organization Guideline Development: An Evaluation

Abstract

BackgroundResearch in 2007 showed that World Health Organization (WHO) recommendations were largely based on expert opinion, rarely used systematic evidence-based methods, and did not follow the organization's own “Guidelines for Guidelines”. In response, the WHO established a “Guidelines Review Committee” (GRC) to implement and oversee internationally recognized standards. We examined the impact of these changes on WHO guideline documents and explored senior staff's perceptions of the new procedures.Methods and FindingsWe used the AGREE II guideline appraisal tool to appraise ten GRC-approved guidelines from nine WHO departments, and ten pre-GRC guidelines matched by department and topic. We interviewed 20 senior staff across 16 departments and analyzed the transcripts using the framework approach. Average AGREE II scores for GRC-approved guidelines were higher across all six AGREE domains compared with pre-GRC guidelines. The biggest changes were noted for “Rigour of Development” (up 37.6%, from 30.7% to 68.3%) and “Editorial Independence” (up 52.7%, from 20.9% to 73.6%). Four main themes emerged from the interviews: (1) high standards were widely recognized as essential for WHO credibility, particularly with regard to conflicts of interest; (2) views were mixed on whether WHO needed a single quality assurance mechanism, with some departments purposefully bypassing the procedures; (3) staff expressed some uncertainties in applying the GRADE approach, with departmental staff concentrating on technicalities while the GRC remained concerned the underlying principles were not fully institutionalized; (4) the capacity to implement the new standards varied widely, with many departments looking to an overstretched GRC for technical support.ConclusionsSince 2007, WHO guideline development methods have become more systematic and transparent. However, some departments are bypassing the procedures, and as yet neither the GRC, nor the quality assurance standards they have set, are fully embedded within the organization.

Nursing Practice Guidelines in China do Need Reform: A Critical Appraisal Using the AGREE II Instrument

“…Its use as a means to critically appraise the quality of CPGs is increasing in the scientific literature (Sabharwal, Patel, Gauher, Holloway, & Athanasiou, ; Sanclemente, Acosta, Tamayo, Bonfill, & Alonso‐Coello, ). Furthermore, it has been recently adopted by WHO for appraising its CPGs (Polus et al., ; Sinclair et al., ).…”

Evidence‐based guidelines for intra‐articular injection in knee osteoarthritis: Formulating and evaluating research questions

“…The CPG will be developed following the new guideline definition from the Institute of Medicine and the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument . We also adhere to the World Health Organization (WHO) handbook for guideline development . The Grading of Recommendations Assessment, Development and Evaluation (GRADE) instrument was used to assess the quality of evidence and to develop recommendations .…”

“…The key clinical questions to be solved by a high‐quality CPG should come from and serve clinical practices in the real world . However, there is no unified methodology currently for framing the important clinical questions at the beginning of developing CPGs .…”

Strength of recommendations in WHO guidelines using GRADE was associated with uptake in national policy

“…HIV and TB guidelines had previously been found to show a high adherence to the GRADE process [4], and exploratory Web searches found a number of repositories that could serve as sources for national guidelines. All HIV and TB guidelines published by WHO from January 2008 to December 2013 were retrieved from a WHO repository (http://who.int/publications/guidelines/en/) and cross-checked with experts at WHO to check that no guidelines were missed.…”

“…WHO adopted the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) [2] system in January 2009 to support the formulation of evidence-based recommendations. Since that time, there has been a steady increase in the number of recommendations issued by WHO that are based on the GRADE approach [3], [4] and [5].…”