Expectations of Behaviorally Anchored Rating Scales

Abstract

Behaviorally anchored rating scales (BARS) are reviewed from two perspectives. First, the particular BARS methodology is assessed on the basis of its fulfilling three broad categories of “criteria for criteria:” (1) utilization criteria, (2) qualitative criteria, and (3) quantitative or psychometric criteria. These three broad categories are composed of 14 specific criteria by which performance evaluation methodologies can be assessed. The second perspective involves an evaluation of BARS in terms of psychometric criteria but with the literature broken down into studies concerned with (1) development of BARS, (2) utilization of BARS, (3) comparison of BARS to other methods, and (4) rater training in BARS use. Conclusions from these reviews indicate that BARS is no better or worse than other methods when assessed on a quantitative basis whereas it has greater potential when assessed on the utilization and qualitative criteria. Suggestions are offered for extending BARS research to process questions and to domains other than performance appraisal perse

Exploring Methods for Developing Behaviorally Anchored Rating Scales for Evaluating Structured Interview Performance

“…Consequently, BARS development has been described many times over the past 50 years, and the following draws liberally on both primary studies (e.g., Bernardin, LaShells, Smith, & Alvares, 1976;Campbell et al, 1973;Hedge, Borman, Bruskiewicz, & Bourne, 2004;Smith & Kendall, 1963) and reviews (e.g., Borman, 1986;Guion, 2011;Jacobs et al, 1980;Schwab et al, 1975).…”

“…Job performance appraisal BARS are often created from the critical incidents obtained through a systematic job analysis using SMEs from a single job, and it is difficult to see how this task could be appropriately "outsourced" to online respondents. Indeed, the source of BARS being job experts' judgments automatically imbues them with job relevance, which is a critical component of legal defensibility in adverse impact cases (Jacobs et al, 1980;Jeanneret & Zedeck, 2010;Landy, Gutman, & Outtz, 2010). Jacobs et al (1980) noted:…”

“…Indeed, the source of BARS being job experts' judgments automatically imbues them with job relevance, which is a critical component of legal defensibility in adverse impact cases (Jacobs et al, 1980;Jeanneret & Zedeck, 2010;Landy, Gutman, & Outtz, 2010). Jacobs et al (1980) noted:…”

“…Edited versions of these critical incidents are used to anchor different levels of the evaluative continuum, providing specific examples of what various levels of performance constitute for each performance dimension. Not only do behavioral examples reduce the construct-irrelevance variance of raters' differing interpretations through provision of common reference points from which to base their judgments (Jacobs, Kafry, & Zedeck, 1980); ease of interpretation is facilitated because the terminology and content of the performance dimensions and behavioral anchors are derived from SMEs who are intimately familiar with the job of interest (Campbell, Dunnette, Arvey, & Hellervik, 1973).…”

“…Edited versions of these critical incidents are used to anchor different levels of the evaluative continuum, providing specific examples of what various levels of performance constitute for each performance dimension. Not only do behavioral examples reduce the construct-irrelevance variance of raters' differing interpretations through provision of common reference points from which to base their judgments (Jacobs, Kafry, & Zedeck, 1980); ease of interpretation is facilitated because the terminology and content of the performance dimensions and behavioral anchors are derived from SMEs who are intimately familiar with the job of interest (Campbell, Dunnette, Arvey, & Hellervik, 1973).Sometimes the dimensions being evaluated are framed in terms of job-relevant psychological attributes (e.g., cooperation, initiative; Borman & Dunnette, 1975) rather than performance dimensions (Kavanagh, 1971). In these circumstances, BARS can be used to restrict idiosyncratic interpretations as well, by focusing raters' attention on concrete behaviors that define those characteristics explicitly in the context of the specific job of interest, rather than abstractly or in "glittering generalities" (Guion, 2011, p. 461); raters are asked to perform less of an inferential leap by focusing on what ratees do at work-i.e., concrete actions-rather than who they are overall-i.e., generalized properties, or traits (cf.…”