Development and test of a new model for validity generalization.

Abstract

Infinite sample-size tests revealed that the Schmidt and Hunter validity generalization procedure underestimates situational variance. A new multiplicative model was developed. The independent version of the multiplicative model was found to be conservative in that the variance of true validities was consistently overestimated. Overall, the results indicate that the multiplicative model provides good accuracy without risking overgeneralization. A retrospective analysis showed that part of the reason for the inaccuracy of the Schmidt-Hunter procedure was that the estimation equation was mathematically incorrect. Subsequent modifications of the estimation equation and the more recent computational procedures employed by Schmidt, Hunter, Pearlman, and Shane; by Pearlman, Schmidt, and Hunter; and by Schmidt, Gast-Rosenberg, and Hunter are also discussed. When applied to three test cases, it was found that the Schmidt, Hunter, Pearlman, and Shane method was substantially more accurate than the original Schmidt-Hunter method but still erred somewhat by underestimating the variance of true validities in all instances with imperfect criterion reliabilities. The most recent Schmidt independent procedure was found to give underestimates in some cases and overestimates in others. A new Schmidt dependent model was found to consistently yield conservative overestimates of variance and was only slightly less accurate than the independent multiplicative model. Extrapolating from these results, we infer that the variance estimates given in Schmidt, Hunter, Pearlman, and Shane may underestimate the true validity variance slightly; those in Schmidt, Gast-Rosenberg, and Hunter and Pearlman, Schmidt, and Hunter probably exceed actual variance slightly.Requests for reprints should be sent to John C. __j IT,,_(.~_ _«:,,t~,j -,,* tkot ™,,^v, o«^ ",-.£, Callender, who is now at the Department of Personnel *£ H " nte 5 P°m' ed OUt *at much and pOS-Administration, Procter and Gamble Company, P.O. slbl V a11 of the between-Study variation in Box 599, Cincinnati, Ohio 45201.observed validity coefficients may be arti-

A Meta-Analysis of Organizational Complaint Handling and Customer Responses

“…As opposed to other meta-analyses in marketing (e.g., Kirca, Jayachandran, and Bearden 2005;Palmatier et al 2006), the Fisher z transformation was not used here for two reasons. Using simulations, Callender and Osburn (1980) show that the Fisher z transformed correlation coefficients severely underestimate the true variance of the correlation coefficients. Moreover, Field (2001) shows that using the Fisher z transformation for heterogeneous effect sizes overestimates the true effect size by about 15% to 45%.…”

Validity Generalization and Situational Specificity: An Analysis of the Prediction of First—Year Grades in Law School

“…In particular, estimation procedures presented by Callender and Osburn (1980) and those used by Pearlman et al (1980) were compared. These results were compared with results based on &dquo;bare-bones validity generalization&dquo; (Schmidt et al, in press), i.e., corrections for sampling error alone.…”

Employee Selection in Times of Change

“…To assess how predictive the scores are of the criterion of interest (i.e., job performance) in personnel selection, the observed correlation should be corrected for range restriction and criterion unreliability. Typically, meta-analytic methods (Callender and Osburn, 1980;Cooper and Hedges, 1994;Hunter and Schmidt, 2004) are used to estimate the criterion-related validity of the predictors used in personnel selection. Such meta-analyses also test whether validities generalize across jobs and settings and are therefore also referred to as validity generalization studies.…”