Development and Scaling of a Preservice Teacher Rating Instrument

Hartsough, Carolyn S.; Perez, Katherine Diana; Swain, Carole L.

doi:10.1177/0022487198049002006

Cited by 3 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BARS have proven to be popular far beyond the original job performance domain for which they were originally developed and have been used to assess classroom teamwork (Ohland et al, 2012), motivation (Landy & Guion, 1970), teaching effectiveness (Eley & Stecher, 1997;Hartsough, Perez, & Swain, 1998), morale (Motowidlo & Borman, 1977), and personality traits (Muck, Hell, & Höft, 2008). Due to their highly specific, highly work-relevant content, BARS have also been suggested as being a viable basis for creating organizational feedback and training programs (Blood, 1974;Campbell et al, 1973;Hom, DeNisi, Kinicki, & Bannister, 1982).…”

Section: Behaviorally Anchored Rating Scalesmentioning

confidence: 99%

“…In these circumstances, BARS can be used to restrict idiosyncratic interpretations as well, by focusing raters' attention on concrete behaviors that define those characteristics explicitly in the context of the specific job of interest, rather than abstractly or in "glittering generalities" (Guion, 2011, p. 461); raters are asked to perform less of an inferential leap by focusing on what ratees do at work-i.e., concrete actions-rather than who they are overall-i.e., generalized properties, or traits (cf. Anastasi, 1938;Cantor, 1990;Schwab et al, 1975), potentially leading to a decrease in illusory halo error (Borman, 1991).BARS have proven to be popular far beyond the original job performance domain for which they were originally developed and have been used to assess classroom teamwork (Ohland et al, 2012), motivation (Landy & Guion, 1970), teaching effectiveness (Eley & Stecher, 1997;Hartsough, Perez, & Swain, 1998), morale (Motowidlo & Borman, 1977, and personality traits (Muck, Hell, & Höft, 2008). Due to their highly specific, highly work-relevant content, BARS have also been suggested as being a viable basis for creating organizational feedback and training programs (Blood, 1974;Campbell et al, 1973;Hom, DeNisi, Kinicki, & Bannister, 1982).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Exploring Methods for Developing Behaviorally Anchored Rating Scales for Evaluating Structured Interview Performance

Kell

Martín‐Raugh

Carney

et al. 2017

ETS Research Report Series

View full text Add to dashboard Cite

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.The Daniel Eignor Editorship is named in honor of Dr. Daniel R. Eignor, who from 2001 until 2011 served the Research and Development division as Editor for the ETS Research Report series. The Eignor Editorship has been created to recognize the pivotal leadership role that Dr. Eignor played in the research publication process at ETS. ETS Research Report Series ISSN 2330-8516 R E S E A R C H R E P O R T Exploring Methods for Developing Behaviorally Anchored Rating Scales for Evaluating Structured Interview PerformanceHarrison J. Kell, Michelle P. Martin-Raugh, Lauren M. Carney, Patricia A. Inglese, Lei Chen, & Gary FengEducational Testing Service, Princeton, NJ Behaviorally anchored rating scales (BARS) are an essential component of structured interviews. Use of BARS to evaluate interviewees' performance is associated with greater predictive validity and reliability and less bias. BARS are time-consuming and expensive to construct, however. This report explores the feasibility of gathering participants' responses to structured interview questions through an online crowdsourcing platform and using those responses to develop BARS. We describe the development of 12 structured interview questions to assess four applied social skills, elicitation of responses to these questions in the form of critical incidents from 68 respondents, and the creation of BARS from these critical incidents. Results indicate online participants are able to produce responses of sufficient quality to generate BARS for evaluating structured interview performance. We conclude by discussing limitations to this approach and future directions for research and practice.Keywords Amazon Mechanical Turk; behaviorally anchored rating scales; crowdsourcing; employment interviews; performance appraisal; social skills; structured interviews doi:10.1002/ets2.12152 Employment interviews are one of the most popular means of selecting personnel (Levashina, Hartwell, Morgeson, & Campion, 2014;McDaniel, Whetzel, Schmidt, & Maurer, 1994). Structured interviews that present all interviewees with the same standardized questions have higher validities for pre...

show abstract