The effect of rater accountability on performance ratings: A meta-analytic review

Harari, Michael B.; Rudolph, Cort W.

doi:10.1016/j.hrmr.2016.09.007

Cited by 29 publications

(31 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, the Across Borders theme is alive and well, but there remains the need for more attention on the expatriate (Claus and Briscoe, 2009) On the other hand, the 'measurement' research is far from dead and buried as predicted by Lawler (1994), and there remains a captivation with performance appraisal, accuracy and ratings (Adler et al, 2016;Chiang and Birtch, 2010;Iqbal et al, 2015). Issues such as conscious rating bias (see Spence and Keeping, 2011) and rater accountability (Harari and Rudolph, 2017) have once again emerged. According to Brutus (2010), the process of evaluating individual performance in organisations hinges on the use of numerical ratings, a proven yet relatively narrow operationalization of this process.…”

Section: ) Across Bordersmentioning

confidence: 99%

Employee performance management: charting the field from 1998 to 2018

Maley

Dabić

Moeller³

2020

IJM

View full text Add to dashboard Cite

PurposeThe purpose of this paper is to study the development and consider the future of one of the most controversial areas of human resource development – employee performance management (PM).Design/methodology/approachThrough bibliometrics, a multiple correspondence analysis identifies the main research directions of PM studies and provides a map of descriptors and a list of authors, along with a framework to track PM literature over 20 years (1998–2018).FindingsScholars have attempted to address some of the questions raised by earlier researchers. However, critical questions remain unanswered, and there is increasing dissatisfaction with the process. The most glaring yet unaddressed problem with PM is poor employee acceptability of the process.Practical implicationsIf the research gaps are addressed, the lack of acceptability of the PM could be resolved and more effectively managed in the future.Originality/valueThe study particularly addresses poor employee “acceptability” of the PM process, a subject that has received limited attention by scholars.

show abstract

Section: ) Across Bordersmentioning

confidence: 99%

Employee performance management: charting the field from 1998 to 2018

Maley

Dabić

Moeller³

2020

IJM

View full text Add to dashboard Cite

show abstract

“…We carried out a systematic literature review of experimental studies on accountability, to disclose insights from the behavioral sciences to a public administration audience. Our focus is on experimental studies because the effects of accountability on judgements, attitudes, and behaviors in the behavioral sciences have been predominantly studied using experimental designs (Hall, Frink, & Buckley, 2017;Harari & Rudolph, 2017;Lerner & Tetlock, 1999). While there is a long tradition of using experimentation in the behavioral sciences, public administration scholars have less experience with this method (Bouwman & Grimmelikhuijsen, 2016;James, Jilke, & Ryzin, 2017), even with the recent rise in interest in behavioral public administration (Grimmelikhuijsen, Jilke, Olsen, & Tummers, 2017).…”

mentioning

confidence: 99%

Lessons from five decades of experimental and behavioral research on accountability: A systematic literature review

Aleksovska

Schillemans

Grimmelikhuijsen

2019

JBPA

View full text Add to dashboard Cite

The study of accountability in public administration has developed largely in parallel to the study of accountability in the behavioral sciences. In an effort to bridge this divide, we present a systematic review of the experimental literature on accountability in the behavioral sciences and draw lessons for public administration. We summarize the findings of 266 experiments exploring the effects of accountability mechanisms, presented in 211 articles published between 1970 and 2016. These findings are organized in four broad themes: effects of accountability on decision-making, behavior, and outcomes; and effects of the specific characteristics of accountability mechanisms. The review shows numerous desirable effects of accountability on individual decision-making and behaviors. This is of high relevance to public administration studies on accountability as it sheds light on causal mechanisms and allows for a balanced perspective on positive and negative effects of various types of accountability mechanisms. It is however not always possible to translate findings from behavioral research directly to public administration settings. We discuss the meaning and value of our findings for public administration studies and develop an agenda for future behavioral research on public sector accountability.

show abstract

“… 10. Although there is ample evidence that managers dislike performance appraisals (e.g., Pulakos et al, 2015), PAs are also likely to elicit a wide range of motivations that result in purposeful managerial behavior. Mechanisms such as accountability (e.g., Harari & Rudolph, 2017), calibration meetings (Speer, Tenbrink, & Schwendeman, 2019), and the naturally complex combination of social, self-serving, and organizationally driven motives (e.g., Longenecker, Sims, & Gioia, 1987; Spence & Keeping, 2013) all make the task of giving operational appraisals a purposeful one. …”

mentioning

confidence: 99%

Scoring Dimension-Level Job Performance From Narrative Comments: Validity and Generalizability When Using Natural Language Processing

Speer

2020

Organizational Research Methods

View full text Add to dashboard Cite

Performance appraisal narratives are qualitative descriptions of employee job performance. This data source has seen increased research attention due to the ability to efficiently derive insights using natural language processing (NLP). The current study details the development of NLP scoring for performance dimensions from narrative text and then investigates validity and generalizability evidence for those scores. Specifically, narrative valence scores were created to measure a priori performance dimensions. These scores were derived using bag of words and word embedding features and then modeled using modern prediction algorithms. Construct validity evidence was investigated across three samples, revealing that the scores converged with independent human ratings of the text, aligned numerical performance ratings made during the appraisal, and demonstrated some degree of discriminant validity. However, construct validity evidence differed based on which NLP algorithm was used to derive scores. In addition, valence scores generalized to both downward and upward rating contexts. Finally, the performance valence algorithms generalized better in contexts where the same qualitative survey design was used compared with contexts where different instructions were given to elicit narrative text.

show abstract

The effect of rater accountability on performance ratings: A meta-analytic review

Cited by 29 publications

References 53 publications

Employee performance management: charting the field from 1998 to 2018

Employee performance management: charting the field from 1998 to 2018

Lessons from five decades of experimental and behavioral research on accountability: A systematic literature review

Scoring Dimension-Level Job Performance From Narrative Comments: Validity and Generalizability When Using Natural Language Processing

Contact Info

Product

Resources

About