Construct validity of a personality assessment game in a simulated selection situation and the moderating roles of the ability to identify criteria and dispositional insight

Abstract

There is scant research on the validity of personality assessment games in selection situations. Therefore, in two experimental simulated selection studies, the construct validity of an assessment game developed to assess honesty‐humility was tested. Both studies found no differences between a control condition and a simulated selection condition on honesty‐humility game scores. Moreover, convergent and discriminant validity with self‐reported personality were not affected by the manipulation. We obtained mixed evidence that individual differences in dispositional insight and the ability to identify criteria influenced the validity of the game. As the validity of the personality assessment game was not significantly affected in the simulated selection context, our findings may imply that well‐designed personality assessment games can be used for high‐stakes selection assessments.

Does understanding what a test measures make a difference? On the relevance of the ability to identify criteria for situational judgment test performance

“…So, one implication is that—contrary to findings in the realm of assessment centers and situational interviews (Ingold et al, 2015; Jansen et al, 2013; Kleinmann et al, 2011; König et al, 2007)—ATIC may be of little relevance in the herein applied set of SJT items (but also see Melchers & Hupp, 2017; Oostrom et al, 2016). In fact, the herein reported correlations were more similar to those found in studies in which ATIC was correlated with personality assessments (Barends & de Vries, 2023; Holtrop et al, 2021; König et al, 2006), 9 thus potentially challenging the view of SJTs as simulations (e.g., Krumm et al, 2015).…”