Comparison of Male vs Female Resident Milestone Evaluations by Faculty During Emergency Medicine Residency Training

Abstract

IMPORTANCE Although implicit bias in medical training has long been suspected, it has been difficult to study using objective measures, and the influence of sex and gender in the evaluation of medical trainees is unknown. The emergency medicine (EM) milestones provide a standardized framework for longitudinal resident assessment, allowing for analysis of resident performance across all years and programs at a scope and level of detail never previously possible. OBJECTIVE To compare faculty-observed training milestone attainment of male vs female residency training DESIGN, SETTING, AND PARTICIPANTS This multicenter, longitudinal, retrospective cohort study took place at 8 community and academic EM training programs across the United States from July 1, 2013, to July 1, 2015, using a real-time, mobile-based, direct-observation evaluation tool. The study examined 33 456 direct-observation subcompetency evaluations of 359 EM residents by 285 faculty members. MAIN OUTCOMES AND MEASURES Milestone attainment for male and female EM residents as observed by male and female faculty throughout residency and analyzed using multilevel mixed-effects linear regression modeling. RESULTS A total of 33 456 direct-observation evaluations were collected from 359 EM residents (237 men [66.0%] and 122 women [34.0%]) by 285 faculty members (194 men [68.1%] and 91 women [31.9%]) during the study period. Female and male residents achieved similar milestone levels during the first year of residency. However, the rate of milestone attainment was 12.7% (0.07 levels per year) higher for male residents through all of residency (95% CI, 0.04-0.09). By graduation, men scored approximately 0.15 milestone levels higher than women, which is equivalent to 3 to 4 months of additional training, given that the average resident gains approximately 0.52 levels per year using our model (95% CI, 0.49-0.54). No statistically significant differences in scores were found based on faculty evaluator gender (effect size difference, 0.02 milestone levels; 95% CI for males, −0.09 to 0.11) or evaluator-evaluatee gender pairing (effect size difference, −0.02 milestone levels; 95% CI for interaction, −0.05 to 0.01). CONCLUSIONS AND RELEVANCEAlthough male and female residents receive similar evaluations at the beginning of residency, the rate of milestone attainment throughout training was higher for male than female residents across all EM subcompetencies, leading to a gender gap in evaluations that continues until graduation. Faculty should be cognizant of possible gender bias when evaluating medical trainees.

Does Applicant Gender Have an Effect on Standardized Letters of Evaluation Obtained During Medical Student Emergency Medicine Rotations?

“…Studies from undergraduate and graduate medical education have demonstrated gender differences associated with several assessment methods. [18][19][20][21][22][23][24][25][26][27][28] For example, men in EM residencies achieve higher milestone assessments than women at graduation. 20 With the revised format of the SLOE 2 and the tendency for more SLOEs to be completed by groups of EM residency and clerkship leaders, 9,15 rather than individual faculty, we sought to determine if there is any difference in the SLOE global assessment of female versus male applicants.…”

Gender effects in medical education: implications for training and gender equity in regional anesthesia

“…Dayal et al 30 compared more than 33 000 milestone evaluations from 359 emergency medicine residents in eight institutions, showing that male residents had a higher rate of milestone attainment throughout training despite men and women starting at similar levels in PGY1. This occurred over all 23 competencies, with the largest difference seen in ‘procedural skills’ and ‘airway’.…”

Gender Bias in Simulation-Based Assessments of Emergency Medicine Residents

“…A recent study of milestonebased, end-of-shift evaluations of EM residents showed a gender disparity, 12 with men advancing to higher levels of competency more quickly than women.…”

“…One recent large multi-site cohort study of EM residents evaluated bias in end-ofshift evaluations and found a significant gender bias based on resident gender. 12 This may be because shift evaluations usually represent subjective assessments. Residents are evaluated only on the cases they saw during a particular shift, resulting in considerable variation with respect to which competencies were assessed across residents and rated by faculty.…”