Development and validation of a HEXACO situational judgment test

Oostrom, Janneke K.; Vries, Reinout E. de; Wit, Mariska de

doi:10.1080/08959285.2018.1539856

Cited by 42 publications

(58 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, SMEs with job experience in the domain in question are typically relied on to form SJT scoring keys (Motowidlo et al, 1990). However, SJTs are also used to measure more generalizable individual differences such as emotional intelligence (MacCann & Roberts, 2008) and personality (Oostrom, de Vries, & de Wit, 2019). Thus, depending on the extent to which an SJT measures job-specific knowledge versus more generalizable individual difference constructs, SME-derived scoring keys may be comparable to those produced by large groups of novices.…”

mentioning

confidence: 99%

Can you crowdsource expertise? Comparing expert and crowd-based scoring keys for three situational judgment tests

Brown¹,

Grossenbacher²,

Martín‐Raugh³

et al. 2020

Preprint

View full text Add to dashboard Cite

It is common practice to rely on a convenience sample of subject matter experts (SMEs) when developing a scoring key for situational judgment tests (SJTs). However, the defining characteristics of what constitutes a SME are often ambiguous and inconsistent across studies. Other research fields have adopted crowdsourcing methods to replace or reproduce judgments thought to require subject matter expertise. Therefore, we conducted the current study to compare crowdsourced scoring keys to SME-based scoring keys for three SJTs in different domains, each varying in job-relatedness. Our results indicate that scoring keys derived from crowdsourced samples are likely to converge with keys based on SME judgment, regardless of test content (correlations ranging from r = .88 to .94). We observed the weakest agreement among individual MTurker and SME ratings for the more job-specific Medical SJT (classification consistency = 61%) but the aggregate scoring keys remained highly correlated. We observed stronger agreement in response option rankings for the Military and Communication SJTs (80% and 85%), which were both designed to require less procedural knowledge. Although general mental ability and conscientiousness were each related to greater expert similarity among MTurkers, the average crowd rating outperformed nearly all individual MTurk raters. Based on an analysis of randomly-drawn bootstrapped samples of MTurker ratings in each of the three samples, we found that as few as 30-40 raters may provide adequate estimates of SME judgments of most SJT items. We hope that these findings help inspire others to consider using crowdsourcing methods as an alternative to SMEs.

show abstract

mentioning

confidence: 99%

Can you crowdsource expertise? Comparing expert and crowd-based scoring keys for three situational judgment tests

Brown¹,

Grossenbacher²,

Martín‐Raugh³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Note that only the Emotional stability and the Conscientiousness scales showed alpha coefficients below .70. The alpha coefficients of these two scales are similar to or even higher than the alpha coefficients of other personality scales with a similar number of items per scale, including the brief version of the HEXACO‐PI‐R (BHI; De Vries, ) and the HEXACO‐SJT (Oostrom et al, ).…”

Section: Methodsmentioning

confidence: 62%

“…However, as time is often a limited resource and long measures increase the likelihood of fatigue, boredom, and burden among participants (Romero, Villar, Gómez‐Fraguela, & Lopez‐Romero, ), interest in developing short measures has grown considerably (Credé, Harms, Niehorster, & Gaye‐Valentine, ). Importantly, there is more and more evidence showing that the effects on validity of short personality measures with generally low alpha coefficients are not as large as previously feared (De Vries, ; Gosling, Rentfrow, & Swann, ; Oostrom, De Vries, & De Wit, ; Romero et al, ). Therefore, we believe future PO fit studies might profit most from a short commensurate personality measure.…”

Section: Introductionmentioning

confidence: 94%

The criterion‐related validity of a short commensurate measure of personality‐based person‐organization fit

Iyer

Oostrom

Serlie

et al. 2019

Int J Selection Assessment

Self Cite

View full text Add to dashboard Cite

Our goal was to develop a commensurate measure of personality‐based person‐organization fit and examine its criterion‐related validity for two retention‐related attitudes: job satisfaction and intention to stay. At T0, 637 employees completed our questionnaire. Two years later, at T1, 171 of the original respondents completed our questionnaire again. Results showed that the organizational personality perceptions are relatively stable. Polynomial regression analyses and surface plots revealed that job satisfaction and intention to stay are higher when there is perfect fit between the person and the organization compared to when the person and the organization differ in traits. However, in most cases, we found the work attitudes to be maximized when both the person and the organization score high on the personality traits.

show abstract

“…Samples for the SJT items and BDI questions are shown in Table 2. Past studies on personality-based SJTs have reported internal consistencies between α = 0.55 and α = 0.75 (Mussel et al, 2018), and between α = 0.22 and α = 0.66 (Oostrom et al, 2019). Past studies on personality-based BDIs reported ICCs (interrater reliability) of 0.78 (Heimann et al, 2020) and 0.74 (Van Iddekinge et al, 2005).…”

Section: Sample Questionmentioning

confidence: 99%

“…To date, few studies have developed personality SJTs or BDIs and even fewer have measured established personality traits such as the Big Five (Goldberg, 1992 ). The few studies that exist, however, suggest that SJTs and BDIs might be useful for measuring personality (Van Iddekinge et al, 2005 ; Oostrom et al, 2019 ; Heimann et al, 2020 ). Accordingly, more research on complementary measurements of personality is needed to foster this initial evidence and to systematically compare these new measures with each other.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Personality Assessment in the Selection Context: A Study Protocol on Alternative Measures and an Extended Bandwidth of Criteria

et al. 2021

View full text Add to dashboard Cite

Personality traits describe dispositions influencing individuals' behavior and performance at work. However, in the context of personnel selection, the use of personality measures has continuously been questioned. To date, research in selection settings has focused uniquely on predicting task performance, missing the opportunity to exploit the potential of personality traits to predict non-task performance. Further, personality is often measured with self-report inventories, which are susceptible to self-distortion. Addressing these gaps, the planned study seeks to design new personality measures to be used in the selection context to predict a wide range of performance criteria. Specifically, we will develop a situational judgment test and a behavior description interview, both assessing Big Five personality traits and Honesty-Humility to systematically compare these new measures with traditional self-report inventories regarding their criterion-related validity to predict four performance criteria: task performance, adaptive performance, organizational citizenship behavior, and counterproductive work behavior. Data will be collected in a simulated selection procedure. Based on power analyses, we aim for 200 employed study participants, who will allow us to contact their supervisors to gather criterion data. The results of this study will shed light on the suitability of different personality measures (i.e., situational judgment tests and behavior description interviews) to predict an expanded range of performance criteria.

show abstract

Development and validation of a HEXACO situational judgment test

Cited by 42 publications

References 76 publications

Can you crowdsource expertise? Comparing expert and crowd-based scoring keys for three situational judgment tests

Can you crowdsource expertise? Comparing expert and crowd-based scoring keys for three situational judgment tests

The criterion‐related validity of a short commensurate measure of personality‐based person‐organization fit

Enhancing Personality Assessment in the Selection Context: A Study Protocol on Alternative Measures and an Extended Bandwidth of Criteria

Contact Info

Product

Resources

About