IntroductionCompetency-based assessment of resident performance requires faculty who are prepared to provide these assessments. Studies have shown that end-of-rotation evaluations completed by faculty lack reliability and fail to identify important deficiencies in residents' performance.
1,2A number of factors account for the less-than-acceptable reliability, including variations in clinical context or patient complexity, 3 lack of longitudinal experiences with residents and patients, 4 and rater error. Types of rater error include leniency error, in which residents' abilities are rated higher than performance merits; halo or reverse halo, when performance in 1 domain affects ratings in other domains; range restriction, in which ratings fall within a narrow range; and error of undifferentiation, in which raters fail to differentiate among different performance domains. [5][6][7][8] In 2009 a Milestone Task Force of the Accreditation Council for Graduate Medical Education and American Board of Internal Medicine (ABIM) published its draft Milestone document, consisting of a list of 142 ''curricular'' Milestones (which are distinct from the 22 internal medicine reporting Milestones).9,10 These curricular Milestones represent specific, observable skills in the 6 competencies residents should demonstrate at each level of training during the 3 years of training. Although program directors have developed new assessment tools based on the Milestones, little is known about how best to educate and support faculty in using the new tools. To date, research has not examined the effect of faculty development in the use of a Milestone-based evaluation form. We sought to determine whether faculty development would reduce common rating errors among faculty completing Milestone-based end-of-rotation evaluations. Our hypothesis was that faculty who participated in faculty development
AbstractBackground Rater errors, such as halo/reverse halo, range restriction, and leniency errors, are frequently cited as threats to the validity of resident assessment by faculty.