Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines

Abstract

Background: The AGREE instrument has been validated for evaluating Clinical Practice Guidelines (CPG) pertaining to medical care. This study evaluated the reliability and validity of physical therapists using the AGREE to assess quality of CPGs relevant to physical therapy practice.

Inconsistencies in Guidelines for Visual Health Surveillance of VDT Workers

“…Measurement error and variation in the perspective of the appraisers involved and the limited number of appraisers were other disadvantages; however, the guidance for AGREE recommends that each set of guidelines be assessed by at least two and preferably four appraisers to obtain good reliability. Studies comparing reliability for a different number of appraisers have found that the largest increments were obtained when going from two to three appraisers 26) . Low item and domain scores for the three sets of guidelines and the agreement between appraisers indicate where improvements are necessary.…”

“…The highest overall kappa coefficient was found for appraisers 1 and 3, which could be due to their similar professional profiles, related with occupational medicine and nursing, as opposed to appraiser 2, an optometrist with expertise in visual ergonomics. It should be kept in mind that some domains such as Stakeholder involvement or Contribution may be appraised using different criteria depending on the appraiser's clinical or applied experience 26) . Thus, the variability observed among appraisers in these domains may reflect a combination of measurement error and variation in perspective.…”

“…Inter-rater reliability in the appraisal of each of the three sets of guidelines was evaluated by calculating intraclass correlation coefficients (ICC) and the agreement between each pair of appraisers by the kappa coefficient (two categories were created: 0, meaning strongly disagree and disagree, and 1, meaning agree and strongly agree). ICC or kappa values above 0.75, between 0.40 and 0.75 and less than 0.40 were considered to represent good, moderate and poor reliability, respectively 26) . Differences between scores of each pair of appraisers for each domain (appraisers 1 and 2, appraisers 1 and 3 and appraisers 2 and 3) were also calculated, representing the percentage of these differences by domain.…”

Towards a Standardized Approach for Critical Care Nursing Practices: Development of Evidence Based Analgo-Sedation Guidelines for Adult Mechanically Ventilated Patients

“…[30] Also, the AGREE II tool has been tested for reliability and validity, and is applicable to a wide variety of health professionals. [31] …”

Evaluation of the Practice Guidelines of Finnish Institute of Occupational Health with AGREE Instrument

“…It has been used in various evaluations with great success. A wide range of people from different cultural and professional backgrounds have found it useful and easy to use 15) . The AGREE instrument can be downloaded free of charge in several languages from the Internet: (http://www.agreecollaboration.org/) 13,16,17) .…”