A critical review of the use of cognitive ability testing for selection into graduate and higher professional occupations

Abstract

This article presents a critical review of the use of cognitive ability testing for access to graduate and higher professional occupations to promote further debate and reflection in both the academic and practitioner community. The main contentions are that the practice of applying cognitive ability testing in these contexts has strong potential to both maintain and exacerbate social inequality in access to higher occupations and professions, and that validity evidence does not justify this to the extent that has previously been presumed. Five critical observations are examined, namely (1) evidence of adverse impact in test outcomes; (2) the tendency to position cognitive ability testing early in selection processes in high‐volume recruitment; (3) recent evidence challenging the meta‐analytic validity of cognitive ability tests; (4) weaknesses in historical primary validity studies; (5) conceptually flawed examination of differential validity evidence in the literature. Implications for practice are discussed, contrasting strategies that involve modifying selection systems that include cognitive testing, versus removing and replacing cognitive tests.

The Performance of Large Language Models on Quantitative and Verbal Ability Tests: Initial Evidence and Implications for Unproctored High-stakes Testing

“…Even prior to the threat of candidates using LLMs to complete tests on their behalf, the use of cognitive ability testing for selection has recently been called into question by academics due to concerns around the initial over-estimation of their criterion-related validity (Sackett et al, 2023;Sackett et al, 2022;cf., Schmidt & Hunter, 1998) and the adverse impact these tests can have on disadvantaged group members (e.g., Woods & Patterson, 2023). Our results suggest that LLMs have created new problems for unproctored testing, particularly for verbal ability tests, given that they can perform better than most human test takers and are widely available for people to use.…”

Racism underlies seemingly race‐neutral conservative criticisms of DEI statements among Black and White people in the United States

“…Perhaps Black participants were more suspicious of (or less enthusiastic about) requests for DEI statements for several reasons. Black people may perceive these initiatives as threatening in the absence of other safety cues (e.g., Marshburn et al, 2021) and organizational initiatives (e.g., the use of cognitive ability tests) that may contribute to inequality (Woods & Patterson, 2023). Black people and people from other marginalized groups may also reasonably question why primarily White organizations are asking them to credential themselves as diverse.…”