A longitudinal meta-analysis of range restriction estimates and general mental ability validity coefficients: Better addressing overcorrection amid decline effects.

Abstract

Psychometric corrections can be crucial for obtaining valid operational results, but concerns are rising about potential overcorrections for general mental ability (GMA) validity coefficients. Our two-part study identifies a source of overprediction: using national norms rather than recent local applicant pool variance for range restriction corrections. Study 1 demonstrates increasing homogeneity in Wonderlic occupational applicant pool variance across four data time waves, suggesting they are no longer interchangeable with the general working population, a divergence attributable to a rise in education. Study 2 employs the Morris meta-analytic approach to gauge the impact of using national norms over occupational ones in range restriction. An analysis of 649 GMA validity coefficients from four time waves of General Aptitude Test Battery and Wonderlic data shows a radical drop in corrected and uncorrected correlations, indicating that historical corrected GMA validity coefficients differ from contemporary ones by up to 16-fold (i.e., an R 2 of 42.3% vs. 2.6%), and range restriction corrections are now minimal in about 75% of cases. This drop in correlations appears due to the filtering effects of increased education, both due to the demands of the knowledge economy and credentialism, where organizations are using college or university degrees as a proxy for GMA. Credentialism is an incredibly inefficient form of GMA assessment, suggesting an urgent societal need to incorporate selection fundamentals more broadly. Altogether, these results indicate that labor market dynamics have a deeper impact on personnel selection than typically appreciated, meaning that many of our estimates have and will eventually age out.