The Effects of Task Information Training and Frame-of-Reference Training With Situational Constraints on Rating Accuracy

Keown-Gerrard, Janine L.; Sulsky, Lorne M.

doi:10.1207/s15327043hup1404_2

Cited by 12 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, task performance assessment revealed that the scores of both the group that received Training_4_Programmes and the group trained in FOR and PDimT were closer to those issued by the group of experts than the control group and other experimental groups. This result concurs with that of previous studies, where participants trained in the creation of a common frame-of-reference, either independently or combined with other types of training, gave more accurate appraisals than those who had received no training or minimal training [46,33,34,39,41,14,15,29,47,35,48,36,37,49,27,25,50,18,32,51,52,28,16,17,53,54]. The results of this study, unlike those obtained in other studies [23,7], show that training in Rater Error included in a combined programme does not affect appraisal accuracy.…”

Section: Discussionsupporting

confidence: 92%

“…By contrasting with the fourth hypothesis (1.4 and 2.4), we have attempted to decide whether task and citizenship performance appraisal varies according to the type of training received. Several studies have analyzed the effectiveness of the various types of rater training along these lines [56,14,57,35,27,50,32,13,52,16,53,54,7]. Likewise, in order to evaluate the improvement of assessment accuracy, most studies [58,14,15,18,16,17,54,19] have used Cronbach’s indices [11] or Borman’s distance accuracy index [12].…”

Section: Discussionmentioning

confidence: 99%

“…In recent years, the FOR programme has been the most used and cited by various authors compared with other strategies [33,34,14,15,35,36,37,18,13,38,28,16], although variations have been observed in the studies regarding experimental design, programme structure, length of training, and method of accuracy appraisal. Studies have shown the positive effect of the FOR programme on appraisal accuracy [39,14,40,27,13,28,16].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Does effectiveness in performance appraisal improve with rater training?

2019

View full text Add to dashboard Cite

Performance appraisal is a complex process by which an organization can determine the extent to which employees are performing their work effectively. However, this appraisal may not be accurate if there is no reduction in the impact of problems caused by possibly subjective rater judgements. The main objective of this work is to check the effectiveness—separately and jointly—of the following four training programmes in the extant literature aimed at improving the accuracy of performance assessment: 1) Performance Dimension Training, 2) Frame-of-Reference, 3) Rater Error Training, and 4) Behavioural Observation Training. Based on these training strategies, three programmes were designed and applied separately. A fourth programme was a combination of the other three. We analyzed two studies using different samples (85 students and 42 employees) for the existence of differences in the levels of knowledge of performance and its dimensions, rater errors, observational accuracy, and accuracy of task and citizenship performance appraisal, according to the type of training raters receive. First, the main results show that training based on performance dimensions and the creation of a common framework, in addition to the training that includes the four programmes (Training_4_programmes), increases the level of knowledge of performance and its dimensions. Second, groups that receive training in rater error score higher in knowledge of biases than the other groups, whether or not they have received training. Third, participants’ observational accuracy improves with each new moment measure (post-training and follow-up), though not because of the type of training received. Fourth, participants who receive training through the programme that combine the other four gave a task performance appraisal that was closer to the one undertaken by the judges-experts than the other groups. And finally, students’ citizenship performance appraisal does not vary according to type of training or to different moment measures, whereas the group of employees who received all four types of training gave a more accurate citizenship performance assessment.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Does effectiveness in performance appraisal improve with rater training?

2019

View full text Add to dashboard Cite

show abstract

“…We averaged over the 'other' manipulation. Also, some studies, such as Sulsky, Skarlicki, and Keown-Gerrard (2002) and Keown-Gerrard and Sulsky (2001), introduced a revised version of FOR training and compared this revised version to both a traditional FOR training and a control session. We ignored all revised versions of FOR training and only report results based on traditional FOR training.…”

Section: Meta-analytic Proceduresmentioning

confidence: 99%

Rater training revisited: An updated meta‐analytic review of frame‐of‐reference training

Roch

Woehr

Mishra

et al. 2011

J Occupat & Organ Psyc

163

149

View full text Add to dashboard Cite

The present study updates Woehr and Huffcutt's (1994) rater training meta-analysis and demonstrates that frame-of-reference (FOR) training is an effective method of improving rating accuracy. The current meta-analysis includes over four times as many studies as included in the Woehr and Huffcutt meta-analysis and also provides a snapshot of current rater training studies. The present meta-analysis also extends the previous meta-analysis by showing that not all operationalizations of accuracy are equally improved by FOR training; Borman's differential accuracy appears to be the most improved by FOR training, along with behavioural accuracy, which provides a snapshot into the cognitive processes of the raters. We also investigate the extent to which FOR training protocols differ, the implications of protocol differences, and if the criteria of interest to FOR researchers have changed over time.The evaluation of individual work performance is a long-standing and ubiquitous concern for both organizational researchers and practitioners alike. To date, there have literally been thousands of published articles focusing on performance appraisal (PA) (a cursory search of SSCI using 'performance appraisal' as the search term revealed 2,887 hits). Moreover, an overwhelming proportion of this literature has focused on ways to improve the quality of evaluations provided by individual raters. In general, two approaches have been pursued with respect to improving performance ratings: developing better rating scales and training raters. Of these two approaches, rater training is by far the most successful and widely accepted. In fact, Landy and Farr (1980) called for a moratorium on research investigating rating scales because scale differences accounted for only a small percentage of the variance in ratings. On the other hand, training raters can result in large improvements in rating accuracy (e.g., Woehr & Huffcutt, 1994). Moreover, even though very little empirical research has investigated the benefits of rater training

show abstract

“…Historically, two rater training programs have received the most attention: Rater error training (RET) that aims to teach trainees to recognize and avoid common distributional rating errors, such as leniency, central tendency and halo (Latham, Wexley, & Pursell, 1975) and frame-of-reference training (FORT) that provides trainees with a common conceptualization of performance as a basis for observing and evaluating ratee behavior (Bernardin & Buckley, 1981). However, in recent years, FORT has dominated rater training research because of its reliable success in enhancing rating accuracy (Gorman & Rentsch, 2009;Keown-Gerrard & Sulsky, 2001;Melchers, Lienhardt, von Aarburg, & Kleinmann, 2011;Roch et al, 2012;Schleicher, Day, Mayes, & Riggio, 2002;Uggerslev & Sulsky, 2008). For this reason, FORT has increasingly been used for rater training in many areas where subjective judgments are used, including competency modeling (Lievens & Sanchez, 2007) assessment centers (Goodstone & Lopez, 2001;Jackson, Atkins, Fletcher, & Stillman, 2005;Mulder, Jorgensen, Nel, & Miering, 2013;Schleicher et al, 2002), and interviews (Melchers et al, 2011;Mills, 1999).…”

Section: Introductionmentioning

confidence: 99%

Rater training: Understanding effects of training content, practice ratings, and feedback

Hauenstein

McCusker

2017

Int J Selection Assessment

View full text Add to dashboard Cite

Despite the popularity of frame‐of‐reference training (FORT), it is not clear how different structural elements of FORT work in concert to improve rating accuracy. Furthermore, past rater training studies have lacked rigorous control groups leading to low thresholds for showing improvements in rating accuracy due to FORT. The current study allowed for the isolation of components of rater training that increase rating accuracy when compared to a rigorously designed control group. Results indicated that repeated rendering of practice ratings improve rating accuracy and this practice effect was amplified by practice rating feedback. Although accuracy‐based training content improved interrater agreement, it did not contribute to improvements rating accuracy over and above the control group. We discuss the implications of the findings in relation to best practices for designing rater training programs.

show abstract

The Effects of Task Information Training and Frame-of-Reference Training With Situational Constraints on Rating Accuracy

Cited by 12 publications

References 25 publications

Does effectiveness in performance appraisal improve with rater training?

Does effectiveness in performance appraisal improve with rater training?

Rater training revisited: An updated meta‐analytic review of frame‐of‐reference training

Rater training: Understanding effects of training content, practice ratings, and feedback

Contact Info

Product

Resources

About