The situational judgement test: a student&amp;rsquo;s worst nightmare

Najim, Muhammad; Rabee, Riham; Sherwani, Yusuf; Ahmed, Maroof; Ashraf, Muhammad; Al-Jibury, Osama; Najim, Rula

doi:10.2147/amep.s95023

Cited by 8 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, they provide evidence to support the intuitive sense, expressed by some, that excessive weight may be given to the SJT scores within the ranking process. 16 Rather our results would suggest a 2:1 weighting ratio between the EPM decile and the SJT score may be the optimum, if the aim is to rank and advantage candidates most likely to successfully complete the foundation programme. Policywise, our findings provide less evidence to support using the educational achievements score in calculating applicant rankings for the foundation programme.…”

Section: Discussionmentioning

confidence: 72%

“…Najim et al noted that an applicant could jeopardise 5–6 years of hard work at medical school through underperformance on a single, 2-hour test. 16 With the advent of the UK Medical Education Database (UKMED), it is now possible to link information from the various databases used to administer medical education. 17 Thus, there was an opportunity to assess the extent to which the selection scores predicted foundation ARCP outcomes for two cohorts of applicants.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study

Smith

Tiffin

2018

BMJ Open

View full text Add to dashboard Cite

ObjectivesCurrently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria.MethodsData were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants’ performance on the three selection measures to the odds of successful programme completion were modelled.ResultsOn univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95% CI 1.10 to 1.18, p<0.001) and SJT z-score decile (OR 1.05, 95% CI 1.01 to 1.09, p=0.02).ConclusionsThe EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.

show abstract

Section: Discussionmentioning

confidence: 72%

Section: Introductionmentioning

confidence: 99%

Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study

Smith

Tiffin

2018

BMJ Open

View full text Add to dashboard Cite

show abstract

“…68 Indeed the weight placed on SJTs, relative to other measures, in medical selection has been debated in relation the allocation process for UK medical graduates to be placed on the country's Foundation training programme (the first 2 years of postgraduate medical training). 70,71 In this case equal weight is currently given to the scores from a 2.4 hour long SJT and the education percentile measure (EPM) derived from academic performance in the previous 5 to 6 years of medical school. 72 Nevertheless, it should be highlighted that, in this situation, the focus of discussion has not been with the quality of the specific SJT used, but rather the relative weight placed on the scores.…”

Section: Implications For Policy and Practicementioning

confidence: 99%

Situational judgement test validity for selection: A systematic review and meta‐analysis

et al. 2020

View full text Add to dashboard Cite

Context Situational judgement tests (SJTs) are widely used to evaluate ‘non‐academic’ abilities in medical applicants. However, there is a lack of understanding of how their predictive validity may vary across contexts. We conducted a systematic review and meta‐analysis to synthesise existing evidence relating to the validity of such tools for predicting outcomes relevant to interpersonal workplace performance. Methods Searches were conducted in relevant databases to June 2019. Study quality and risk of bias were assessed using the Quality In Prognosis Studies (QUIPS) tool. Results were pooled using random effects meta‐analysis and meta‐regressions. Results Initially, 470 articles were identified, 218 title or abstracts were reviewed, and 44 full text articles were assessed with 30 studies meeting the final inclusion criteria and were judged, overall, to be at moderate risk of bias. Of these, 26 reported correlation coefficients relating to validity, with a pooled estimate of 0.32 (95% confidence interval 0.26 to 0.39, P < .0001). Considerable heterogeneity was observed (I2 = 96.5%) with the largest validity coefficients tending to be observed for postgraduate, rather than undergraduate, selection studies (β = 0.23, 0.11 to 0.36, P < .001). The correction of validity coefficients for attenuation was also independently associated with larger effects (β = 0.13, 0.03 to 0.23, P = .01). No significant associations with test medium (video vs text format), cross‐sectional study design, or period of assessment (one‐off vs longer‐term) were observed. Where reported, the scores generally demonstrated incremental predictive validity, over and above tests of knowledge and cognitive ability. Conclusions The use of SJTs in medical selection is supported by the evidence. The observed trend relating to training stage requires investigation. Further research should focus on developing robust criterion‐relevant outcome measures that, ideally, capture interpersonal aspects of typical workplace performance. This will facilitate additional work identifying the optimal place of SJTs within particular selection contexts and further enhancing their effectiveness.

show abstract

“…10,13 We agree that interviews (either structured or multiple-mini interviews) are an important part of the selection process for both the candidate and the employer to assess person-organisation fit, 14 and to allow candidates to provide rationales for their given responses. However, while it is important to retain face-to-face methodologies in a selection process, using interviews in lieu of SJTs would represent a plethora of challenges in terms of both practicalities and validity.…”

Section: Sjts Are Efficient and Cost-effectivementioning

confidence: 96%

The value of situational judgement tests for assessing non-academic attributes in dental selection

Taylor

Mehra²,

Elley³

et al. 2016

Br Dent J

View full text Add to dashboard Cite

The situational judgement test: a student’s worst nightmare

Cited by 8 publications

References 3 publications

Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study

Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study

Situational judgement test validity for selection: A systematic review and meta‐analysis

The value of situational judgement tests for assessing non-academic attributes in dental selection

Contact Info

Product

Resources

About