Can Basic Individual Differences Shed Light on the Construct Meaning of Assessment Center Evaluations?

Abstract

The construct meaning of assessment center evaluations is an important unresolved issue in I/O psychology. This study hypothesized that Cognitive Ability and personality traits are primary correlates of evaluators' overall assessment ratings (OARs). Metaanalysis results based on 65 correlations indicate the following mean construct-level correlations with OARs: .67 for Cognitive Ability, .50 for Extraversion, .35 for Emotional Stability, .25 for Openness, and .17 for Agreeableness; yielding a multiple R of .84. These findings support our hypothesis and cast light on the construct meaning of assessment center evaluations.

Assessment Center Dimensions: Individual differences correlates and meta‐analytic incremental validity

“…First, we provide the first large-scale investigation of the overlap between AC dimensions with individual differences traits (cognitive ability and personality) in two independent samples (total N ¼ 4985). The sample sizes available in our primary data exceed those of prior investigations of overall AC ratings, even of metaanalytic ones (e.g., Collins et al, 2003;Scholz and Schuler, 1993). Second, we integrate the findings from our primary data with meta-analytic predictive validity estimates for ACs, personality, cognitive ability, as well as meta-analytic data on the interrelationships of these predictors.…”

“…Correlations from the mid-level and top-manager samples are presented separately as well as combined (sample size weighted). For comparative purposes, the correlations between cognitive ability and the Big 5 with overall AC ratings from Collins et al (2003) and Scholz and Schuler (1993) are also given in Table 1. The correlational pattern was remarkably similar across the two samples of managers, and hence only the results for the combined sample are discussed here.…”

“…These results are presented in Table 5; again, details on the computations can be found in Appendix Tables A3-A5. The operational validity of the Big 5 as a set, cognitive ability, and the Big 5 and cognitive ability combined were estimated at .29, .56, and .62 based on the available meta-analytic evidence. Incremental validities were computed based on the correlations with AC dimensions obtained from this study and the metaanalytic correlations with overall AC ratings from Collins et al (2003) and Scholz and Schuler (1993). Results show that cognitive ability and the Big 5 as a set add substantial value over all types of AC ratings.…”

“…Data that speak to this matter are available for overall AC ratings. Collins et al (2003) meta-analytically examined the nomological net of overall AC ratings that represent aggregates of several constructs. Collins and colleagues' meta-analysis provides information on the overlap of overall AC ratings with tests of cognitive ability ( r ¼ :43) and four of the Big 5 dimensions of personality (agreeableness, extraversion, emotional stability, and openness; r ¼ :12, .36, .26, and .18, respectively).…”

“…Similarly, the meta-analytic estimates provided by Collins et al (2003) can be used to determine that the incremental validity of overall AC ratings over psychometric tests of personality and cognitive ability is negligible. However, the lack of correlational evidence for AC dimensions hinders parallel investigations for construct-based AC scores.…”