Good News: Work Sample Administrators' Global Performance Judgments are (About) as Valid as We've Suspected

Revised Estimates of Dimension and Exercise Variance Components in Assessment Center Postexercise Dimension Ratings.

“…(p. 118). We speculate that AC performance is best characterized as consisting of two broad components: (a) general, but situation-specific, performance factors corresponding to overall performance in each exercise qua work sample task (Lance, Johnson, Douthitt, Bennett, & Harville, 2000;Smith, 1991;Teachout & Pellum, 1991), and (b) a crosssituationally stable overall performance factor that is driven by stable traits such as cognitive ability, conscientiousness, and experience.…”

“…Sackett and Tuzinski (2001) took an alternative view, stating that “assessment centers do not ‘lack construct validity,’ but rather lack clear consensus as to the constructs they do assess … at issue is whether the traditional explanation of what constructs assessment centers measure is the correct one.” (p. 118). We speculate that AC performance is best characterized as consisting of two broad components: (a) general, but situation-specific, performance factors corresponding to overall performance in each exercise qua work sample task (Lance, Johnson, Douthitt, Bennett, & Harville, 2000; Smith, 1991; Teachout & Pellum, 1991), and (b) a cross-situationally stable overall performance factor that is driven by stable traits such as cognitive ability, conscientiousness, and experience.…”

“…We included all studies analyzed previously by Lievens and Conway (2001), plus five others that met their criteria for inclusion that were not available to them previously (matrices reported by Lance et al 2000, Lance, Foster, Gentry, & Thorensen, 2004).…”

Assessor Cognitive Processes in an Operational Assessment Center.

“…Zedeck (1986) proposed that these overall evaluations are guided by assessors' schemas for candidate performance. In addition, Lance, Newbolt, et al (2000) speculated that, despite being trained to evaluate assessee performance according to the prescribed dimensions, assessors may instead “regard exercises as work sample tests” (p. 347), and Lance, Johnson, Douthitt, Bennett, and Harville (2000) showed, reassuringly, that work sample administrators' global performance judgments are quite accurate. As such, assessors may first evaluate how an assessee performed overall in the exercise (qua work sample) and then assign dimensional ratings on the basis of their overall evaluations of the assessee's performance in the exercise.…”

Personnel selection: A primer.

“…Second, they tend to suffer from validity problems. Specifically, correlations among dimensions within an exercise tend to be higher than correlations across exercises measuring a given dimension (Lance, Johnson, Douthitt, Bennett, & Harville, 2000). Using the above example, it should be that a person's interpersonal skills score from the sales presentation correlates strongly with a person's interpersonal skills score from the leaderless group discussion (i.e., the person either has these skills or not).…”