Managerial Functions: An Alternative to Traditional Assessment Center Dimensions?

Abstract

The construct validity of traditional assessment center dimensions was compared with that of a set of alternative constructs based on the functional structure of managerial work. Subjects were 75 middle‐level managers in state government who participated in two developmental assessment centers as part of a centralized management development program. One assessment center measured performance in terms of traditional attribute dimensions and the other in terms of functions performed in managerial work. Results show that evidence for construct validity is weak for both sets of constructs.

Constructs and developmental assessment centers: Further troubling empirical findings

“…For the four data sets that could be inverted, Sackett and Harris found that the ratings were dominated by exercise factors. Several other studies (e.g., Joyce et al, 1994;Silverman et al, 1986;Sackett & Dreher, 1982;Tarnage & Muchinsky, 1982) support these findings. Overall, the evidence strongly suggests that assessment center ratings are measuring situational-specific performance rather than cross-situational managerial abilities.…”

“…They concluded that the dimension ratings did not accurately reflect the abilities they were purported to measure. Several other studies support these findings (e.g., Joyce, Thayer & Pond, 1994;Sackett & Dreher, 1982;Schneider & Schmitt, 1992;Silverman, Dalessio, Woods & Johnson, 1986;Turnage & Muchinsky, 1982). According to Bycio et al, the failure of ACs to demonstrate construct validity may result from the cognitive limitations of the assessors to rate accurately a large number of complex dimensions.…”

“…Silverman et al (1986) described a within-dimension method of producing AC ratings, which resulted in some improvement in their construct validity. Joyce et al (1994) investigated an AC that was designed using managerial functions rather than traditional dimensions; however, they did not find any improvement in construct validity for the alternative method.…”

“…Given the amount of evidence indicating the opposite, some redesign of the assessment center method may be needed. Joyce et al (1994) discuss possible reasons for the failure of AC dimensions to show construct validity. These reasons include dimension intercorrelation, exercise design, and the rating process.…”

Design And Analysis Of Incomplete Multitrait-Multimethod Studies From A Multiplicative Perspective

“…Marsh (1990) considered an incomplete MTMM design and stated that incomplete designs are likely to happen in research based on existing personality instruments. Incomplete MTMM designs also arise frequently in assessment center studies (Arthur, Woehr, & Maldegen, 2000;Bobrow & Leonards, 1997;Harris, Becker, & Smith, 1993;Joyce, Thayer, & Pond, 1994;Lance, Foster, Gentry, & Thoresen, 2004;Lievens & van Keer, 2001;Kudisch, Ladd, & Dobbins, 1997). An assessment center evaluates job candidates' skills and abilities using several methods that are available for the purpose.…”