Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study

Abstract

Traditional methods of assessing personality traits in medical school selection have been heavily criticised. To address this at the point of selection, ''non-cognitive'' tests were included in the UK Clinical Aptitude Test, the most widely-used aptitude test in UK medical education (UKCAT: http://www.ukcat.ac.uk/). We examined the predictive validity of these non-cognitive traits with performance during and on exit from medical school. We sampled all students graduating in 2013 from the 30 UKCAT consortium medical schools. Analysis included: candidate demographics, UKCAT non-cognitive scores, medical school performance data-the Educational Performance Measure (EPM) and national exit situational judgement test (SJT) outcomes. We examined the relationships between these variables and SJT and EPM scores. Multilevel modelling was used to assess the relationships adjusting for confounders. The 3343 students who had taken the UKCAT non-cognitive tests and had both EPM and SJT data were entered into the analysis. There were four types of non-cognitive test: (1) libertariancommunitarian, (2) NACE-narcissism, aloofness, confidence and empathy, (3) MEARS-self-esteem, optimism, control, self-discipline, emotional-nondefensiveness (END) and faking, (4) an abridged version of 1 and 2 combined. Multilevel regression showed that, after correcting for demographic factors, END predicted SJT and EPM decile. Aloofness and empathy in NACE were predictive of SJT score. This is the first large-scale study examining the relationship between performance on non-cognitive selection tests and medical school exit assessments. The predictive validity of these tests was limited, and the relationships revealed do not fit neatly with theoretical expectations. This study does not support their use in selection.

Assessments of Non-academic Attributes in Applicants for Undergraduate Medical Education: an Overview of Advantages and Limitations

“…These findings support the widely held agreement by medical educators that the validity, fairness, and cost-effectiveness of MMIs, SJTs, SCs, and personality traits require further study and development [36,46]. Selection of medical students based on their academic achievements, but not on measures of non-academic attributes, is not only consistent with presently available data; it might also lead to substantial savings in costs and manpower.…”

Selection methods in healthcare professions: where are we now and where next?

“…Patterson et al 2015), integrity (Husbands et al 2015) and many other non-academic attributes and skills. MacKenzie et al (2016) paper in this special issue takes steps towards eliminating some of the attributes and traits that are not necessarily suitable to assess at the point of selection into medical school, as they are not found to be predictive of subsequent performance. The authors found these to include narcissism, aloofness, confidence, optimism, control and self-discipline among others.…”

Response distortion on personality tests in applicants: comparing high-stakes to low-stakes medical settings

“…Despite their importance, it remains unclear how best to measure these characteristics, particularly in the context of high-stakes testing where applicants may be motivated to distort their responses (Albanese et al, 2003;Bore et al, 2009;Musson, 2009;Patterson et al, 2016). In recent years, a variety of selection tools have been developed that aim to assess these noncognitive characteristics, including situational judgment tests (Bore et al, 2009;De Leng et al, 2017;Lievens, 2013;Patterson et al, 2012;Patterson et al, 2009), multiple mini-interviews (Eva et al, 2014;Eva et al, 2009;Eva et al, 2004;Griffin et al, 2012a;Kulasegaram et al, 2010), emotional intelligence tests (Libbrecht et al, 2014), and personality tests (Griffin et al, 2012b;MacKenzie et al, 2017;Rothstein et al, 2006). The current study focuses on personality testing in the context of the high-stakes selection of medical interns assessing the degree to which response distortion might limit the utility of personality tests.…”

“…A related important issue is whether response distortion reduces the predictive validity of personality test scores (Griffin et al, 2012b;MacKenzie et al, 2017;Rothstein et al, 2006). The Big Five personality traits have provided a useful organizing framework, with meta-analytic research, largely in non-medical student contexts, finding modest correlations with academic grades for conscientiousness (r = .19) and openness, r = .10, (Poropat, 2009).…”