The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self- and Peer Evaluation

Abstract

Instructors often incorporate self-and peer evaluations when they use teamwork in their classes, which is common in management education. However, the process is often time consuming and frequently does not match well with guidance provided by the literature. We describe the development of a web-based instrument that efficiently collects and analyzes self-and peer-evaluation data. The instrument uses a behaviorally anchored rating scale to measure team-member contributions in five areas based on the team effectiveness literature. Three studies provide evidence for the validity of the new instrument. Implications for management education and areas for future research are discussed.

Exploring Methods for Developing Behaviorally Anchored Rating Scales for Evaluating Structured Interview Performance

“…BARS have proven to be popular far beyond the original job performance domain for which they were originally developed and have been used to assess classroom teamwork (Ohland et al, 2012), motivation (Landy & Guion, 1970), teaching effectiveness (Eley & Stecher, 1997;Hartsough, Perez, & Swain, 1998), morale (Motowidlo & Borman, 1977), and personality traits (Muck, Hell, & Höft, 2008). Due to their highly specific, highly work-relevant content, BARS have also been suggested as being a viable basis for creating organizational feedback and training programs (Blood, 1974;Campbell et al, 1973;Hom, DeNisi, Kinicki, & Bannister, 1982).…”

“…In these circumstances, BARS can be used to restrict idiosyncratic interpretations as well, by focusing raters' attention on concrete behaviors that define those characteristics explicitly in the context of the specific job of interest, rather than abstractly or in "glittering generalities" (Guion, 2011, p. 461); raters are asked to perform less of an inferential leap by focusing on what ratees do at work-i.e., concrete actions-rather than who they are overall-i.e., generalized properties, or traits (cf. Anastasi, 1938;Cantor, 1990;Schwab et al, 1975), potentially leading to a decrease in illusory halo error (Borman, 1991).BARS have proven to be popular far beyond the original job performance domain for which they were originally developed and have been used to assess classroom teamwork (Ohland et al, 2012), motivation (Landy & Guion, 1970), teaching effectiveness (Eley & Stecher, 1997;Hartsough, Perez, & Swain, 1998), morale (Motowidlo & Borman, 1977, and personality traits (Muck, Hell, & Höft, 2008). Due to their highly specific, highly work-relevant content, BARS have also been suggested as being a viable basis for creating organizational feedback and training programs (Blood, 1974;Campbell et al, 1973;Hom, DeNisi, Kinicki, & Bannister, 1982).…”

Student Perceptions on the Impact of Formative Peer Team Member Effectiveness Evaluation in an Introductory Design Course

“…38 The development, reliability, and validity of the instrument are described in Loughry, Ohland, and Moore 39 and Ohland et al…”

The Rapid Adoption of SMARTER Teamwork Tools: the System for Management, Assessment, Research, Training, Education, and Remediation for Teamwork

“…The peer-evaluation instrument can be previewed at www.CATME.org. Although the tools were developed to help instructors manage teams, 4,5 they can also facilitate research on students' learning and document student outcomes related to learning and the quality of their team experiences. 6 We propose seven empirical studies to measure the effect sizes of the following learning experiences: teamwork training, working in teams, rating teamwork, and giving and receiving feedback.…”