A Meta-Analysis of the Relationship Between Rater Liking and Performance Ratings

Abstract

This meta-analysis reviewed the magnitude and moderators of the relationship between rater liking and performance ratings. The results revealed substantial overlap between rater liking and performance ratings (ρ = .77). Although this relationship is often interpreted as indicative of bias, we review studies that indicate that to some extent the relationship between liking and performance ratings potentially reflects "true" differences in ratee performance. Moderator analyses indicated that the relationship between liking and performance ratings was weaker for ratings of organizational citizenship behaviors, ratings made by peer raters, ratings in nonsales jobs, and ratings made for development; however, the relationship was strong across moderator levels, underscoring the robustness of this relationship. Implications for the interpretation of performance ratings are discussed.Performance evaluation systems are central to a cross-section of talent management functions, such as determining employee compensation and rewards, providing developmental feedback, documenting administrative decisions, succession planning, and reinforcing organizational norms (Cascio & Aguinis, 2005). In fact, Ghorpade and Chen (1995) suggested that performance ratings are "inevitable in all organizations-large and small, public and private, local and multinational" (p. 32). Yet performance appraisals have been the subject of substantial criticism over the years. Indeed, skepticism as to the quality of the information obtained from human evaluations has persisted for nearly as long as the field of psychological measurement (Thorndike, 1925;Wells, 1907). Murphy (2008) succinctly summed up the state of affairs, noting "performance ratings are widely viewed as poor measures of job performance" (p. 148).Over the years, a litany of factors has been proposed to hinder the quality of performance ratings. The overarching theme of this school of thought is that raters introduce performance irrelevant variance into performance ratings because they are either unable or unwilling to provide accurate ratings. Early research attributed low-quality ratings to rater ability (or presumably, lack thereof) and sought to design better scales

Moderating effect of pay dispersion on the relationship between employee share ownership and labor productivity

“…Second, others argue that performance evaluations are an indication of rater liking rather than a measure of objective performance. The more we like someone, the more likely we are to rate their performance favorably (Lefkowitz, ; Sutton, Baldwin, Wood, & Hoffman, ). The evidence from a meta‐analysis suggests that the association between rater liking and leader‐linked performance rating reported a coefficient of 0.77 (Sutton et al, ).…”

“…It is plausible that we tend to like people who are high performers. Evidence based on objective measures of performance offered support for this view (Sutton et al, ). Overall, the evidence indicated that despite the presence of subjectivity in performance evaluations, they are sufficiently merit based so as to be perceived as being fair.…”

“…Second, others argue that performance evaluations are an indication of rater liking rather than a measure of objective performance. The more we like someone, the more likely we are to rate their performance favorably (Lefkowitz, 2000;Sutton, Baldwin, Wood, & Hoffman, 2013).…”

“…The evidence from a meta-analysis suggests that the association between rater liking and leader-linked performance rating reported a coefficient of 0.77 (Sutton et al, 2013). However, the direction of causality is debatable.…”

Trick or trait? The combined effects of employee impression management modesty and trait modesty on supervisor evaluations

“…Given the subjective and retrospective nature of this evaluation process, the assessment of an employee's professional and personal qualities are closely intertwined: Prior research has shown a strong positive relationship between supervisors' liking for employees and job performance ratings; in fact, a meta-analysis by Sutton, Baldwin, Wood, and Hoffman (2013) revealed a corrected correlation of ρ = .72 between both constructs. Based on these findings, we expect supervisor job performance ratings to reflect not only the assessment of an employee's competence as a performance-related construct, but also the interpersonal evaluation of an employee's perceived likeability.…”