Subgroup Differences in Situational Judgment Test Performance: A Meta-Analysis

Whetzel, Deborah L.; McDaniel, Michael A.; Nguyen, Nhung T.

doi:10.1080/08959280802137820

Cited by 142 publications

(202 citation statements)

References 32 publications

Supporting

Mentioning

175

Contrasting

Unclassified

Order By: Relevance

“…This was in line with previous research showing that in SJTs, females slightly outperform males (Lievens 2013;Whetzel et al 2008). A possible explanation for the difference between females and males in performance on the difference scenarios could be that females generally score higher on personality factors such as conscientiousness and agreeableness, and some scenarios might relate to these factors more strongly than others (Whetzel et al 2008). We found age effects for two out of six scenarios, on which older applicants scored better than younger applicants.…”

Section: Discussionsupporting

confidence: 80%

“…Additionally, adverse impact of gender has been reported: female applicants tend to score slightly better in SJTs than male applicants (Lievens 2013;Whetzel et al 2008). However, the influence of factors other than gender and ethnicity on SJT performance in medical school selection has not yet been studied.…”

Section: Introductionmentioning

confidence: 99%

“…Research on SJT use in medical student selection shows that SJT scores differ between white individuals and other ethnic groups: white applicants score better than applicants from other ethnic backgrounds (Whetzel et al 2008). Additionally, adverse impact of gender has been reported: female applicants tend to score slightly better in SJTs than male applicants (Lievens 2013;Whetzel et al 2008).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Schripsema

Trigt

Borleffs

et al. 2017

Adv in Health Sci Educ

View full text Add to dashboard Cite

Situational Judgement Tests (SJTs) are increasingly implemented in medical school admissions. In this paper, we investigate the effects of vocational interests, previous academic experience, gender and age on SJT performance. The SJT was part of the selection process for the Bachelor's degree programme in Medicine at University of Groningen, the Netherlands. All applicants for the academic year 2015-2016 were included and had to choose between learning communities Global Health (n = 126), Sustainable Care (n = 149), Intramural Care (n = 225), or Molecular Medicine (n = 116). This choice was used as a proxy for vocational interest. In addition, all graduate-entry applicants for academic year 2015-2016 (n = 213) were included to examine the effect of previous academic experience on performance. We used MANCOVA analyses with Bonferroni post hoc multiple comparisons tests for applicant performance on a six-scenario SJT. The MANCOVA analyses showed that for all scenarios, the independent variables were significantly related to performance (Pillai's Trace: 0.02-0.47, p \ .01).

show abstract

Section: Discussionsupporting

confidence: 80%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Schripsema

Trigt

Borleffs

et al. 2017

Adv in Health Sci Educ

View full text Add to dashboard Cite

show abstract

“…On cognitive ability tests, White test takers have been shown to score approximately one standard deviation higher than non-White test takers (De Soete et al 2013). A meta-analysis on ethnic subgroup differences across 32 SJTs-mainly originating from the US-showed that White test takers score approximately 0.38 standard deviation higher than Black test takers, 0.24 standard deviation higher than Hispanic test takers and 0.29 standard deviation higher than Asian test takers (Whetzel et al 2008). A Dutch study also found that the ethnic subgroup difference in an integrity SJT score (d = 0.38) was lower than in a cognitive ability test score (d = 0.48) (De Meijer et al 2010).…”

Section: Present Studymentioning

confidence: 99%

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Leng

Stegers‐Jager

Husbands

et al. 2016

Adv in Health Sci Educ

View full text Add to dashboard Cite

Situational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used.

show abstract

“…A second approach to examining the role of constructs in personnel assessments is exemplified by McDaniel and colleagues (e.g., McDaniel, Hartman, Whetzel, & Grubb, 2007;Whetzel, McDaniel, & Nguyen, 2008). Rather than examining the content of judges' ratings or applicant behaviors/responses, "marker tests" are used to examine the saturation/correlation of assessments with scores on particular marker tests of constructs (e.g., personality dimensions, cognitive ability).…”

Section: Marker Test Approachmentioning

confidence: 99%