Updating the trainability tests literature on Black–White subgroup differences and reconsidering criterion-related validity.

Abstract

A number of applied psychologists have suggested that trainability test Black–White ethnic group differences are low or relatively low (e.g., Siegel & Bergman, 1975), though data are scarce. Likewise, there are relatively few estimates of criterion-related validity for trainability tests predicting job performance (cf. Robertson & Downs, 1989). We review and clarify the existing (and limited) literature on Black–White group differences on trainability tests, provide new trainability test data from a recent video-based trainability exam, and present archival data about how trainability test scores relate to cognitive ability, Black–White differences, and job performance. Consistent with hypotheses, our results suggest large correlations of trainability tests with cognitive ability (e.g., .80) and larger standardized ethnic group differences than previously thought (ds of 0.86, 1.10, and 1.21 for 3 samples). Results also suggest that trainability tests have higher validity than previously thought. Overall, our analysis provides a substantial amount of data to update our understanding of the use of trainability tests in personnel selection.

Reviewing, Categorizing, and Analyzing the Literature on Black–White Mean Differences for Predictors of Job Performance: Verifying Some perceptions and Updating/Correcting Others

“…This triangulates with empirical evidence in SJT research (McDaniel et al, 2007; Whetzel et al, 2008) and in AC research (Goldstein et al, 1998, 2001; Hough & Oswald, 2000), as well as narrative conjectures in earlier reviews of high stakes testing (Sackett et al, 2001). Indeed, although not reported above, Roth et al, (2011) recently noted a corrected r of .80 between g‐saturation and d for trainability tests.…”

An Evaluation of Spearman's Hypothesis by Manipulating g Saturation

“…In this line of research, as with the Fagan and Holland tests, applicants receive training and then are evaluated on their knowledge of the trained material. Both Harris () in a set of primary studies and Roth et al () in a broader range of studies, which also incorporated the Harris data, reported that such measures show high correlations with g and mean racial differences comparable to those found on g tests. The discrepancy in these two research streams has not been resolved.…”

“…Fagan and Holland (, , ) reported that when Whites and Blacks had a similar exposure to the language (e.g., words, sayings, similarities, analogies) used in the test, there were only negligible mean racial differences in the processing of the information. We contrast the Fagan and Holland research with research on miniature training and evaluation tests (Harris, ), also called trainability tests (Roth, Buster, & Bobko, ). In this line of research, as with the Fagan and Holland tests, applicants receive training and then are evaluated on their knowledge of the trained material.…”

A multitrait–multimethod study of assessment instruments for complex problem solving

“…7 Typically, trainability tests contain work samples to test behavior that is important for specific jobs (e.g., the job of an electrician) without relying on previous experience in this job (Sackett, 2000). They involve a period of time in which participants can acquire relevant knowledge before performance is assessed (Roth, Buster, & Bobko, 2011), similar to the exploration phase found in CPS tasks . Although important differences between CPS tasks and trainability tests exist (e.g., trainability tests clearly apply to a specific content area, whereas this holds for CPS tasks to a lesser extent), the principle of learning central to trainability tests is also found in the concept of CPS and its assessment instruments.…”