Differential prediction: A methodological artifact?

Abstract

Thirty-one studies involving the validity of employment and training selection procedures for blacks and whites were examined. Each pair of validity coefficients from these studies was determined to be based or not be based on each of the following desirable methodological practices: (a) use of a total number of 50 or more for both black and white subsamples, (b) use of a criterion for research purposes only, and (c) use of a predictor chosen for its potential relationship to the criterion being predicted. From these data, each study was scored for adherence to the three desirable practices. There was a significant relationship between the validity outcome of the studies and the methodological practices score. In addition, in studies in which some validity was obtained, those whose authors reported the study as supporting the differential validity concept received significantly lower methodological practices scores. It was concluded that reported findings of differential prediction can largely be regarded as methodological artifacts.

A historical perspective on diversity ideologies in the United States: Reflections on human resource management research and practice

“…Research on affirmative action provided guidelines for how organizations could best institute affirmative action programs (e.g., Hopkins, 1980). In the area of selection, much of the research was confined to surfacing differences in employment outcomes for blacks and whites (e.g., Newman & Krzystofiak, 1979;Tepstra & Larsen, 1985), as well as differential validity and adverse impact in employment tests and how to overcome these problems (e.g., Bayroff, 1966;Boehm, 1977;Hunter, Schmidt, & Hunter, 1979). Other scholars focused on bias and discrimination in performance ratings and evaluation (e.g., Dipboye, 1985;Landy & Farr, 1980), and still others examined racial differences in affective organizational variables like motivation and job satisfaction (Bhagat, 1979;Gavin & Ewen, 1974).…”

Predictor Validity as Related to Criterion Relevance, Restriction of Range, and Ethnicity

“…For example, Schmidt, (1988) concluded that The Journal of Psychology 1993.127:553-563. four literature surveys (Boehm, 1977;Katzell & Dyer, 1977;O'Connor, Wexley, & Alexander. 1975;Schmidt, Berner, & Hunter, 1973) "have now demonstrated that evidence for single-group validity by race (Black vs. White) does not occur any more frequently than would be expected solely on the basis of chance" (p. 274).…”

Using the Thorndike model to assess the fairness of cognitive ability tests for personnel selection

“…Studies used to assess differential prediction normally use a combination of ability tests, including not only cognitive ability but knowledge and special skills such as clerical speed and accuracy (c.f., Boehm, 1977;Field, Bayley & Bayley, 1977;Grant & Bray, 1970;Gael, Grant & Ritchie, 1975b;Schmidt, Berner & Hunter, 1973). Therefore, these differential prediction studies cannot necessarily be generalized to characterize CATs exclusively.…”

“…The assumption, however, is that both Blacks and Whites are characterized by the same regression line. Although it has been found that the slopes of both the Black and White regression lines are often similar, they frequently have different intercepts (e.g., Boehm, 1977;Field, Bayley & Bayley, 1977;Grant & Bray, 1970;Gael, Grant & Ritchie, 1975b;Ruch, 1972, as cited in Arvey & Faley, 1988;Schmidt, Berner & Hunter, 1973). Furthermore, the intercept of the regression line for Blacks is frequently found to be lower than the regression line for Whites, indicating that the use of a common regression line would over-predict job performance for Blacks according to the Cleary rule.…”