Racial Bias on the Emergency Medicine Standardized Letter of Evaluation

Abstract

Background Studies on components of residency applications have shown evidence of racial bias. The Standardized Letter of Evaluation (SLOE) is an assessment measure for emergency medicine (EM) residency applications and, as more specialties opt to use SLOEs in place of narrative letters of recommendation, understanding bias on standardized assessments is essential. Objective To determine whether there is a difference in rankings on the EM SLOE between underrepresented in medicine (UIM) and non-UIM applicants, White and non-White applicants, and to examine whether differences persist after controlling for other characteristics. Methods The sample was drawn from medical students who applied to EM residency at the study institution in 2019. We compared rankings between UIM and non-UIM students and between students of each individual race/ethnicity and White students, after controlling for United States Medical Licensing Examination Step scores, Alpha Omega Alpha status, type of school (US MD, US DO, internation medical graduate), Medical Student Performance Evaluation class percentile, affiliated program vs visiting clerkship SLOE, gender and the interaction of race/ethnicity and gender, and adjusted for students submitting multiple SLOEs, using ordinal regression. Results There were 1555 applicants to the study institution in 2019; 1418 (91.2%) had a SLOE and self-identified race/ethnicity. After controlling for applicant characteristics, non-UIM students were significantly more likely to be ranked higher than UIM students on “Rank Against Peers,” (OR 1.46, 95% CI 1.03-2.07) and Grade (OR 1.46, 95% CI 1.05-2.04). Conclusions Analysis of EM SLOEs submitted to our institution demonstrates racial bias on this standardized assessment tool, which persists after controlling for other performance predictors.

SLOE: A step in the right direction

“…The prior SLOE format had racial bias within multiple sections, including the GA: non-URM candidates were ranked statistically significantly higher than URM candidates. 2 Alvarez et al 3 had similar findings regarding the GA and also found disparities amongst the qualifications for EM questions. "Quiet" applicants were more likely to be placed in lower categories.…”

Racism and sexism in the trajectory of black female medical students: an integrative review

“…In a hegemonically white educational context, black male and female students are discriminated against and less valued in terms of their intellectual potential than their white peers. This unequal treatment can be seen in the lower scores in admissions processes at American, English and Dutch universities and in letters of recommendation from American universities, as pointed out by Kukulski et al 39 and another study 40 . This discrimination can influence the low educational performance of some students, according to Loue et al 41 .…”

“…Kukulski P, et al 39 , 2022, USA, Quantitative Study II; I Identify differences in the classifications of black and white students.…”

A systematic approach to tackling racism in emergency medicine: A commentary on the executive summary of the SAEM 2022 consensus conference

“…Therefore, we must systematically identify and eliminate barriers to EM training for those who are underrepresented in medicine (URiM) and of historically marginalized identities. Racial bias has infiltrated traditional measures of excellence such as clerkship grading, medical student performance evaluation descriptors, honor societies, and the EM standardized letter of evaluation 15–18 . An overreliance on these biased measures to select trainees has contributed to the creation of a nonrepresentative workforce 19 .…”

“…Racial bias has infiltrated traditional measures of excellence such as clerkship grading, medical student performance evaluation descriptors, honor societies, and the EM standardized letter of evaluation. [15][16][17][18] An overreliance on these biased measures to select trainees has contributed to the creation of a nonrepresentative workforce. 19 Holistic review has gained momentum as an evidence-supported method that more evenly balances the experiences, attributes, and metrics of residency applicants.…”