Do Global Rating Forms Enable Program Directors to Assess the ACGME Competencies?

Abstract

Global rating forms, the tool that program directors use most frequently to document residents' competence, may not be adequate to assess the six general competencies. The results are consistent with earlier published research indicating that physicians view competence in just two broad dimensions, which questions the premise of the six ACGME competencies. Further research is needed to validate and measure six distinct dimensions of clinical competence.

The Development of a Competency-Based Assessment Rubric to Measure Resident Milestones

“…6 Because the ideal evaluation should be real time, relevant and practical, global rating forms will continue to have an important role in assessing residents. Identifiable issues related to resident assessments include inadequate description of evaluation criteria, variations in raters' observations and assessments, unsatisfactory or lack of meaningful feedback, and timeliness of feedback.…”

The State of Evaluation in Internal Medicine Residency

“…Ease of use may explain their popularity despite documented problems with discriminatory ability, reliability, and validity. [11][12][13][14] Given the current disconnect between actual practice and recommended practice in using global rating forms, we suggest either a "culture" change by programs in their use of rating forms or added help by the ACGME to improve the psychometric quality of these forms and train raters on their proper use.…”

Development of the Objective Structured System-Interaction Examination

“…9 To obtain a high degree of reliability, at least 40 surveys are needed for each resident from each type of evaluator. [12][13][14] This is not a reasonable target, given the current construct of most residency programs. In 1 study, no significant correlation was found between the 360-degree evaluation during the first year of residency and the same subject's 4th-year objective measures including OSCE performance, class rank, and United States Medical Licensing Examination (USMLE) scores.…”

“…Finally, little information can be retrieved from these surveys that can be used for constructive feedback. 13,14 Checklists are another approach for evaluating systemsbased practice. By using this method, a checklist of all desired activities that reflect systems-based practice is developed.…”