Unhappiness and dissatisfaction in doctors cannot be predicted by selectors from medical school application forms: A prospective, longitudinal study

Abstract

BackgroundPersonal statements and referees' reports are widely used on medical school application forms, particularly in the UK, to assess the suitability of candidates for a career in medicine. However there are few studies which assess the validity of such information for predicting unhappiness or dissatisfaction with a career in medicine. Here we combine data from a long-term prospective study of medical student selection and training, with an experimental approach in which a large number of assessors used a paired comparison technique to predict outcome.MethodsData from a large-scale prospective study of students applying to UK medical schools in 1990 were used to identify 40 pairs of doctors, matched by sex, for whom personal statements and referees' reports were available, and who in a 2002/3 follow-up study, one pair member was very satisfied and the other very dissatisfied with medicine as a career. In 2005, 96 assessors, who were experienced medical school selectors, doctors, medical students or psychology students, used information from the doctors' original applications to judge which member of each pair of doctors was the happier, more satisfied doctor.ResultsNone of the groups of assessors were significantly different from chance expectations in using applicants' personal statements and the referees' reports to predict actual future satisfaction or dissatisfaction, the distribution being similar to binomial expectations. However judgements of pairs of application forms from pairs of doctors showed a non-binomial distribution, indicating consensus among assessors as to which doctor would be the happy doctor (although the consensus was wrong in half the cases). Assessors taking longer to do the task concurred more. Consensus judgements seem mainly to be based on referees' predictions of academic achievement (even though academic achievement is not actually a valid predictor of happiness or satisfaction).ConclusionAlthough widely used in medical student selection to assess motivation, interest and commitment to a medical career, the personal statement and the referee's report cannot validly be used by assessors, including experienced medical school selectors, to identify doctors who will subsequently be dissatisfied with a medical career.

The effect of primary care fellowship training on career satisfaction, happiness and perceived stress

“…These studies also indicated that fellowship participants were more interested in non-clinical matters, such as research, teaching and leadership. This may be indicative of the general views espoused by other researchers on the role of a wider or more cosmopolitan view on the well-being of individuals 12 13…”

Uncertainties in the selection of applicants for medical school

“…The predictive value of aptitude tests is uncertain. On the one hand, a review of prospective studies in the UK found that A level grades, but not aptitude tests, predicted performance in medical school examinations and part 1 of a postgraduate examination (McManus et al 2005a;McManus et al 2003). On the other hand, the scores obtained on the Medical College Admission Test (MCAT) have been reported to predict the performance on the USMLE step I (Donnon et al 2007) and on the Canadian Licensing Examinations-part II (communication and problem exploration skills) (Kulatunga-Moruzi and Norman 2002b), and regression analyses have indicated that the GPA and the MCAT scores had an independent predictive validity for medical school grades and USMLE scores (Julian 2005) (Table 1).…”

“…These tests are expected to be fair, transparent, evidence-based and legally defensible (McManus et al 2005a). Their predictive value is determined not only by their reliability and validity, but also by the ''pre-test'' or ''prior'' probability of the non-cognitive trait under consideration, i.e., its prevalence in the population to which the tested individual belongs (Galen and Gambino 1975).…”

“…GPAs assess knowledge of academic subjects, such as chemistry and mathematics. Aptitude tests assess a mixture of reasoning, intelligence and culturally acquired knowledge, such as vocabulary and geography (McManus et al 2005a). Non-cognitive traits are inferred from the applicants' performance on interviews and on specific assignments, such as small group discussions of a problem (Collins 1995), multiple mini interviews (MMI) (Eva et al 2004), simulated tutorials (Kulatunga-Moruzi and Norman 2002b), and studying a text about a disease and explaining it to another candidate (ten Cate and Smal 2002), as well as from the applicants' demographic data, their scores on personality tests, their personal statements/essays/autobiographical letters and letters of recommendation.…”

Generalizability of a Composite Student Selection Procedure at a University-Based Chiropractic Program

“…Unfortunately, the overall evidence on most other predictors (often termed 'non-cognitive' predictors) is comparatively scarce 5 and even less convincing. 3,8,[10][11][12][13][14] The admission interview is widely used in medical education 15 and it is probably the best examined of the non-cognitive selection tools. 10 While the admission interview appears to have at least high 'face validity' for testing non-cognitive skills, such as communication skills and interpersonal skills, its reliability is controversial in health science education.…”