A review of the content, criterion-related, and construct-related validity of assessment center exercises.

Hoffman, Brian J.; Kennedy, Colby L.; LoPilato, Alexander C.; Monahan, Elizabeth L.; Lance, Charles E.

doi:10.1037/a0038707

Cited by 51 publications

(49 citation statements)

References 131 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When reviewing the effect sizes across the three samples, the difference was more pronounced on the IB exercise (Sample 1: d = 0.95; Sample 2: d = 0.73; Sample 3: d = 0.56). This finding is in line with our original hypothesis, showing alignment to previous research (Goldstein et al, 1998;Hoffman et al, 2015), although the magnitude of these differences were larger for this study. The implication of this finding is that the IB exercise is potentially unsuitable for organizations operating in culturally diverse settings and highlights the impact of political, economic, and societal factors at work in these environments.…”

Section: Independent Samples T-test and Cohen's Dsupporting

confidence: 94%

“…In addition to accounting for the general effectiveness of participants, research on the GPF in AC ratings indicates, for example, a relationship to cognitive ability (Hoffman et al, 2011) and conscientiousness (Lance et al, 2007). These findings resonate with research demonstrating that cognitive ability predicts candidate performance in AC exercises (Hoffman et al, 2015) and AC dimensions (Meriac et al, 2014). We can therefore assert that the presence of a GPF in AC ratings might reflect cognitive ability to some extent:…”

Section: Gpf In Acssupporting

confidence: 52%

See 1 more Smart Citation

General performance factors and group differences in assessment center ratings

Buckett¹,

Becker

Roodt

2017

JMP

View full text Add to dashboard Cite

Purpose-The purpose of this paper is to establish the extent of general performance factors (GPF) in assessment center (AC) exercises and dimensions. The study further aims to determine if larger GPF contributes to larger ethnic group differences across exercises and dimensions that are more cognitively loaded in an emerging market context. Design/methodology/approach-The authors analyzed data across three independent AC samples (Sample 1: N = 172; Sample 2: N = 281; Sample 3: N = 428). The Schmid-Leiman solution was used to determine the extent of GPF in AC exercises and dimensions. An independent samples t-test and Cohen's d was used to determine the size of ethnic group differences across exercises and dimensions. Findings-The results indicate that GPF is consistently large for the in-basket exercise. Furthermore, dimensions that are more cognitively loaded, such as problem solving, strategic thinking, and business acumen, seem to produce the largest ethnic group differences. Overall, the research indicates that larger GPF is associated with larger ethnic group differences in relation to specific AC dimensions and exercises. Originality/value-The authors add to the literature by investigating the prevalence of a GPF in AC ratings across AC exercises and dimensions. A novel contribution of the research attempts to link the prevalence of a GPF in AC ratings to group membership in South Africa. The study offers an alternative statistical analysis procedure to examine GPF in AC ratings.

show abstract

Section: Independent Samples T-test and Cohen's Dsupporting

confidence: 94%

Section: Gpf In Acssupporting

confidence: 52%

General performance factors and group differences in assessment center ratings

Buckett¹,

Becker

Roodt

2017

JMP

View full text Add to dashboard Cite

show abstract

“…Moreover, B. J. Hoffman et al (2015) found small-to-moderate criterion-related validities associated with individual AC exercises.…”

Section: Exercise-versus Dimension-centric Perspectivesmentioning

confidence: 94%

Everything that you have ever been told about assessment center ratings is confounded.

Jackson

Michaelides

Dewberry

et al. 2016

Journal of Applied Psychology

View full text Add to dashboard Cite

Despite a substantial research literature on the influence of dimensions and exercises in assessment centers (ACs), the relative impact of these two sources of variance continues to raise uncertainties because of confounding. With confounded effects, it is not possible to establish the degree to which any one effect, including those related to exercises and dimensions, influences AC ratings. In the current study (N = 698) we used Bayesian generalizability theory to unconfound all of the possible effects contributing to variance in AC ratings. Our results show that ≤ 1.11% of the variance in AC ratings was directly attributable to behavioral dimensions, suggesting that dimension-related effects have no practical impact on the reliability of ACs.Even when taking aggregation level into consideration, effects related to general performance and exercises accounted for almost all of the reliable variance in AC ratings. The implications of these findings for recent dimension-and exercise-based perspectives on ACs are discussed. When, in the context of selection, appraisal, and development, behavioral criteria are used to evaluate individuals, it is essential that these criteria are measured reliably.Unsurprisingly, therefore, the measurement properties of assessment center (AC) ratings have come under close scrutiny in the applied psychology literature. In ACs, the behavior of jobholders or candidates is sampled across several work-related situations (exercises, e.g., a role play exercise, group discussion, presentation) and is typically assessed by trained assessors in terms of pre-defined behavioral dimensions (e.g., communication skills, teamwork, planning and organizing). As a result of their multifaceted measurement properties, incorporating dimensions, exercises, and assessors, ACs provide a rich source of information about the extent to which work-related behavioral criteria can be reliably measured in a job-relevant setting.Historically, researchers have questioned the extent to which behavioral dimensions are measured reliably in ACs, and have implied that researchers should utilize an exercise-oriented approach to scoring ACs

show abstract

“…Past research that studied the nomological network of AC ratings involved the overall assessment rating (e.g., Collins et al, 2003), final dimension ratings (Dilchert & Ones, 2009;Meriac, Hoffman, & Woehr, 2014;Shore, Thornton, & Shore, 1990), overall exercise ratings (Hoffman, Kennedy, LoPilato, Monahan & Lance, 2015), or latent exercise factors together with the latent general performance factor (Lance et al, 2007) and their relationships with the Big Five and general mental ability (GMA). However, this research allows only limited inferences with regard to the constructrelated validity of the different latent components that constitute AC performance, namely, dimensions, exercises, and general performance (Hoffman et al, 2011) relative to each other.…”

Section: Nomological Network Of Ac Factorsmentioning

confidence: 99%