The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions

Abstract

Problem Situational judgment tests (SJTs) can be used to assess the nonacademic attributes necessary for medical and dental trainees to become successful practitioners. Evidence for SJTs’ predictive validity, however, relates predominantly to selection in postgraduate settings or using video-based SJTs at the undergraduate level; it may not be directly transferable to text-based SJTs in undergraduate medical and dental school selection. This preliminary study aimed to address these gaps by assessing the validity of the UK Clinical Aptitude Test (UKCAT) text-based SJT. Approach Study participants were 218 first-year medical and dental students from four UK undergraduate schools who completed the first UKCAT text-based SJT in 2013. Outcome measures were educational supervisor ratings of in-role performance in problem-based learning tutorial sessions—mean rating across the three domains measured by the SJT (integrity, perspective taking, and team involvement) and an overall judgment of performance—collected in 2015. Outcomes There were significant correlations between SJT scores and both mean supervisor ratings (uncorrected r = 0.24, P < .001; corrected r = 0.34) and overall judgments (uncorrected r s = 0.16, P < .05; corrected r s = 0.20). SJT scores predicted 6% of variance in mean supervisor ratings across the three nonacademic domains. Next Steps The results provide evidence that a well-designed text-based SJT can be appropriately integrated, and add value to, the selection process for undergraduate medical and dental school. More evidence is needed regarding the longitudinal predictive validity of SJTs throughout medical and dental training pathways, with appropriate outcome criteria.

What we measure … and what we should measure in medical education

“…Some of these investigations have even reported data to support the fact that assessment results are predictive of future performance (i.e. establish predictive validity) . Putting aside the validity of these assessments or, more appropriately, the inferences we may wish to make based on assessment scores, there has been a general recognition that, in addition to knowledge, other attributes and skills are required to be successful as a medical student or as a resident (postgraduate trainee).…”

“…establish predictive validity). [17][18][19] Putting aside the validity of these assessments or, more appropriately, the inferences we may wish to make based on assessment scores, there has been a general recognition that, in addition to knowledge, other attributes and skills are required to be successful as a medical student or as a resident (postgraduate trainee). Historically, primarily knowledge-based assessments (e.g.…”

What is the value of Values Based Recruitment for nurse education programmes?

“…Studies focused on the measurement of non‐academic attributes (as mentioned previously: not values) in healthcare education recruitment often have academic performance, rather than actual practice and patient care, as an outcome measure (e.g. Husbands & Dowell, ; Oluwasanjo, Wasser, & Alweis, ; Patterson et al., ). In addition, most of these studies are conducted with applicants to courses in medicine and dentistry (Patterson et al., ).…”

Medizinstudierendenauswahl in Deutschland

“…In Großbritannien werden Integrität, Perspektivenübernahme und Teamfähigkeit mittels eines SJTs als Teil des UKCAT [46] …”