Does effectiveness in performance appraisal improve with rater training?

Abstract

Performance appraisal is a complex process by which an organization can determine the extent to which employees are performing their work effectively. However, this appraisal may not be accurate if there is no reduction in the impact of problems caused by possibly subjective rater judgements. The main objective of this work is to check the effectiveness—separately and jointly—of the following four training programmes in the extant literature aimed at improving the accuracy of performance assessment: 1) Performance Dimension Training, 2) Frame-of-Reference, 3) Rater Error Training, and 4) Behavioural Observation Training. Based on these training strategies, three programmes were designed and applied separately. A fourth programme was a combination of the other three. We analyzed two studies using different samples (85 students and 42 employees) for the existence of differences in the levels of knowledge of performance and its dimensions, rater errors, observational accuracy, and accuracy of task and citizenship performance appraisal, according to the type of training raters receive. First, the main results show that training based on performance dimensions and the creation of a common framework, in addition to the training that includes the four programmes (Training_4_programmes), increases the level of knowledge of performance and its dimensions. Second, groups that receive training in rater error score higher in knowledge of biases than the other groups, whether or not they have received training. Third, participants’ observational accuracy improves with each new moment measure (post-training and follow-up), though not because of the type of training received. Fourth, participants who receive training through the programme that combine the other four gave a task performance appraisal that was closer to the one undertaken by the judges-experts than the other groups. And finally, students’ citizenship performance appraisal does not vary according to type of training or to different moment measures, whereas the group of employees who received all four types of training gave a more accurate citizenship performance assessment.

The Effectiveness of VSTEP.3-5 Speaking Rater Training

“…Obviously, these above-presented results supported the point of researchers such as Smith (1986), Woehr and Huffcutt (1994), Noonan and Sulsky (2001), Roch et al (2011), Rosales Sánchez, Díaz-Cabrera, andHernández-Fernaud (2019), who had advocated and provided evidence for the enhancement of rating quality after rater training. Besides, the findings of small increase in score reliability was in line with several reports on slight improvement of rating accuracy by McIntyre, Smith, and Hassett (1984), Noonan and Sulsky (2001), Roch and O'Sullivan (2003).…”

“…A substantial amount of research in the field of testing and assessment has put an emphasis on rater training (Pulakos, 1986;Woehr & Huffcutt, 1994;Roch & O'Sullivan, 2003;Roch, Woehr, Mishra, & Kieszczynska, 2011; to name but a few) in an attempt for improving the rating, yet the findings about its efficiency seem to be inconsistently documented. Many researchers and scholars posited that RET reduced halo and leniency errors (Latham, Wexley, & Pursell, 1975;Smith, 1986;Hedge & Kavanagh, 1988;Rosales Sánchez, Díaz-Cabrera, & Hernández-Fernaud, 2019). These authors assumed that when raters are more aware of the rating errors they may commit, their ratings are likely to be more accurate.…”

Influence of measurement type and the moment of occurrence of low performance behaviour's on task and citizenship performance appraisal

“…When an organization opts to use tools with global items, data correction could be established, since benevolent bias is likely to occur (Díaz-Vilela et al, 2012). (Rosales et al, 2019;Woehr & Huffcutt, 1994). This paper provides new information about how biases can influence the performance assessment process, depending on when low performance occurs.…”

“…This bias directly affects the performance assessment process, since it reduces the possibility of identifying and rewarding employee performance, and performance validity is therefore diminished (Bretz, Milkovich, & Read, 1992). The study of benevolent bias has been closely linked with training programmes created for raters and with the development and improvement of performance assessment techniques (Díaz-Cabrera et al, 2014;Rosales, Díaz-Cabrera & Hernández-Fernaud, 2019;Woehr & Huffcutt, 1994). Traditionally, a distinction has been made between two assessment methods: 1) absolutes, which involve the assessment of assessees against a universal standard or specific behaviour, such as behaviourally anchored rating scales (BARS, Smith & Kendall, 1963) and behavioural observation scales (BOS, Latham & Wexley, 1977) and 2) comparatives, which require the rater to assess the assessee against other assessees, including, for example, pairing comparisons.…”

The Effects of Different Rater Training Procedures on ESL Essay Raters’ Rating Accuracy

“…Although many studies have compared the effects of different rating procedures, especially in the field of performance appraisal (Ellington & Wilson, 2017;Rosales-Sánchez et al, 2019;Tziner et al, 2000), studies comparing the effects of different rater training procedures in assessing language performances seem scarce. Despite several calls to investigate the effect of procedures used for training language performance raters so that these procedures could be put to best use (Freedman 1981;Furneaux & Rignall, 2002;Hamp-Lyons, 1990;McIntyre 1993;O'Sullivan and Rignall, 2001), only one type of training procedure, i.e.…”