Raters who pursue different goals give different ratings.

Abstract

J. N. Cleveland and K. R. Murphy (1992) suggested that phenomena such as rater errors and interrater disagreements could be understood in terms of differences in the goals pursued by various raters. We measured 19 rating goals of students at the beginning of a semester, grouped them into scales, and correlated these with teacher evaluations collected at the end of the semester. We found significant multiple correlations, both within classes and in an analysis of the pooled sample (adjusting for instructor mean differences, incremental R2 =.08). Measures of rating goals obtained after raters had observed a significant proportion of ratee performance accounted for variance (incremental R2 =.07) not accounted for by measures of goals obtained at the beginning of the semester.

Getting Rid of Performance Ratings: Genius or Folly? A Debate

“…Research on the political aspects of performance appraisal suggests that raters pursue a variety of goals when completing performance appraisals and that these goals substantially influence the ratings they give (Murphy, Cleveland, Skattebo, & Kinney, 2004). Murphy and Cleveland (1995) suggest that these goals include (a) task performance goals-using performance ratings to influence the subsequent performance of ratees; (b) interpersonal goals-using appraisal to maintain or improve interpersonal relations between the supervisor and the subordinates; (c) strategic goals-using appraisal to increase the supervisor's and/or the workgroup's standing in the organization; and (d) internalized goals-goals that are the results of raters' beliefs about how he or she should evaluate performance (e.g., some raters want to convey the image of being a hard case, with high performance standards, like the professor who announces at the beginning of a class that he or she rarely gives "A" grades).…”

Rater’s Intention Towards Appraising Accurately

“…Performance appraisal model (Harris, 1994;Levy & Williams, 2004;Murphy & Cleveland, 1995) and performance appraisal research (Murphy, Cleveland, Skattebo, & Kinney, 2004;Spence & Keeping, 2010;Wong & Kwong, 2007) started to pay attention back to raters' motivation in performance appraisal process. Traditionally, it is assumed that raters were motivated to appraise accurately (Levy & Williams, 2004).…”

Practical Impact of Predictor Reliability for Personnel Selection Decisions

“…intuitions, fine-tuned as they might be, because ratings likely fluctuate with respect to a specific manager and/or point in time due to variables ranging from ratings scale formats, rater capacities (e.g., cognitive limitations of the rater), and rater goals (e.g., strategic decisions of the rater) (Bretz, Milkovich, & Read, 1992;Landy & Farr, 1980;Murphy, Cleveland, Skattebo, & Kinney, 2004;Saal, Downey, & Lahey, 1980).…”