Can I retake it? Exploring subgroup differences and criterion-related validity in promotion retesting.

Abstract

Despite recent interest in the practice of allowing job applicants to retest, surprisingly little is known about how retesting affects 2 of the most critical factors on which staffing procedures are evaluated: subgroup differences and criterion-related validity. We examined these important issues in a sample of internal candidates who completed a job-knowledge test for a within-job promotion. This was a useful context for these questions because we had job-performance data on all candidates (N = 403), regardless of whether they passed or failed the promotion test (i.e., there was no direct range restriction). We found that retest effects varied by subgroup, such that females and younger candidates improved more upon retesting than did males and older candidates. There also was some evidence that Black candidates did not improve as much as did candidates from other racial groups. In addition, among candidates who retested, their retest scores were somewhat better predictors of subsequent job performance than were their initial test scores (rs = .38 vs. .27). The overall results suggest that retesting does not negatively affect criterion-related validity and may even enhance it. Furthermore, retesting may reduce the likelihood of adverse impact against some subgroups (e.g., female candidates) but increase the likelihood of adverse impact against other subgroups (e.g., older candidates).

Change in Cognitive Abilities in Older Latinos

“…Therefore, ethnic differences in retest learning, which have been reported in some prior research (Schleicher, Van Iddekinge, Morgeson, & Campion, 2010; Van Iddekinge, Morgeson, Schleicher, & Campion, 2011; Early et al, 2013), could confound between group comparisons of cognitive aging. We investigated this possibility using two additional modeling approaches.…”

Effects of g‐Loading and Time Lag on Retesting in Job Selection

“…However, in another study, Lievens et al () found that cognitive ability tests were more valid predictors on the second attempt for admission to medical school even though those who pass on the first attempt outperform those who pass on their second. Similarly, Van Iddekinge et al () found that for job knowledge tests given for within‐job promotions, the second attempt (.38) was more valid than the first (.27). The Lievens et al () and Van Iddekinge et al () papers suggest that the reasons for potential validity gains lie in an increase in relevant knowledge for knowledge‐based tests, which can occur through exposure to the test or passive, on the job learning.…”

“…Similarly, Van Iddekinge et al () found that for job knowledge tests given for within‐job promotions, the second attempt (.38) was more valid than the first (.27). The Lievens et al () and Van Iddekinge et al () papers suggest that the reasons for potential validity gains lie in an increase in relevant knowledge for knowledge‐based tests, which can occur through exposure to the test or passive, on the job learning. Simultaneously, irrelevant variables may decrease in importance on a second attempt, such as unfamiliarity with the test, making the second attempt a better estimate of the individual's true ability.…”

“…While there does not appear to be much research on gender differences in retesting specifically, the research that exists favors the ability of women to improve their scores on retests more than men. For example, Van Iddekinge et al () found that women improved their scores more on retesting. Schleicher et al () found that women outgained men on retests for behavioral description, situational and experience interviews, as well as case studies, but not necessarily on written tests such as verbal ability and job knowledge.…”

Computing Pareto‐optimal Predictor Composites for Complex Selection Decisions

“…Information concerning the effect sizes of the GRE subtests can be found in reports by the ETS (), while the effect size of the UGPA can be found in, for example, technical reports by the Law School Admission Council (). However, the data required as input for the analytic method can be found not only in meta‐analytic, but in primary studies as well (Van Iddekinge, Morgeson, Schleicher, & Campion, ).…”