In‐basket Validity: A systematic review

Abstract

In‐baskets are high‐fidelity simulations often used to predict performance in a variety of jobs including law enforcement, clerical, and managerial occupations. They measure constructs not typically assessed by other simulations (e.g., administrative and managerial skills, and procedural and declarative job knowledge). We compiled the largest known database (k = 31; N = 3,958) to address the criterion‐related validity of in‐baskets and possible moderators. Moderators included features of the in‐basket: content (generic vs. job specific) and scoring approach (objective vs. subjective) and features of the validity studies: design (concurrent vs. predictive) and source (published vs. unpublished). Sensitivity analyses assessed how robust the results were to the influence of various biases. Results showed that the operational criterion‐related validity of in‐baskets was sufficiently high to justify their use in high‐stakes settings. Moderator analyses provided useful guidance for developers and users regarding content and scoring.

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

“…Disagreements were resolved by referring back to the studies and discussion between the two authors, until consensus was reached. This level of agreement is similar to the inter-coder agreement found in validity generalization studies (Whetzel and McDaniel, 1988; Whetzel et al, 2014).…”

Meta-analytic Examination of a Suppressor Effect on Subjective Well-Being and Job Performance Relationship

“…Therefore, overall SWB and its two components are valid predictors of performance ratings at work. The correlation is similar or even higher than the correlation found for other wellknown variables related to job performance, such as the Big Five personality dimensions, cognitive abilities, emotional intelligence, the situational judgment test, interviews, and in-basket tests (see, for instance, Aguado et al 2019;Alonso et al, 2015;Alonso et al, 2017;Herde et al, 2019;García-Izquierdo et al, 2012;García-Izquierdo et al, 2020;Joseph & Newman, 2010;Judge et al, 2013;Morillo et al, 2019;Moscoso et al, 2012;Moscoso & Salgado, 2001;Ones et al, 1993;Otero et al, 2020;Ryan & Derous, 2019;Salgado et al (2015), Salgado, 2017;Salgado & Lado, 2018;Salgado & Moscoso, 2019b;Salgado & Tauriz, 2014;Van Rooy & Viswesvaran, 2004;Whetzel et al, 2014). Moreover, as evidenced by 90% credibility values, overall SWB, cognitive SWB, and affective SWB generalize validity across samples, instruments, occupations, organizations, and countries.…”

Getting it done and getting it right: Leader disciplinary reactions to Followers’ moral transgressions are determined by construal level mindset

“…We therefore designed our experimental studies to provide a realistic simulation of actual working relationships and a realistic simulation of an engaging job GETTING IT DONE RIGHT 29 simulation task (Whetzel, et al, 2014). We also replicated our findings in a field setting using organizational leaders in Study 3.…”

“…We used an in-basket exercise. In-basket exercises are popular ways to assess job performance (Whetzel et al, 2014) and have been used extensively in leadership experiments (Hoogervorst et al, 2010). We told participants that they would be involved in testing a new assessment tool stimulating an actual work environment.…”