“…In validity studies, the rating sources of job performance can be the supervisors (e.g., Harris et al, 1995; Campbell and Wiernik, 2015), instructors (e.g., Berges et al, 2018), peers (e.g., Harris and Schaubroeck, 1988; Viswesvaran et al, 2002), and incumbents (e.g., Bang and Reio, 2017; Jyoti and Sharma, 2017; Haider et al, 2018; Rehman and Shahnawaz, 2018). This paper focuses on the interrater reliability of supervisory performance ratings as they are the most frequently used in validity studies and for performance appraisal purposes (Bernardin and Beatty, 1984; Landy and Rastegary, 1989; Viswesvaran et al, 2002; Campbell and Wiernik, 2015).…”