Standardized Letters of Recommendation in Plastic Surgery: The Impact of Gender and Race

Abstract

Background: In the last few decades, standardized letters of recommendation have seen increasing popularity among various medical specialties and have been shown to demonstrate score inflation and gender bias in some select specialties. Through analysis of standardized letter of recommendation scoring patterns, this study evaluated the impact of race and gender on performance in standardized letters of recommendation and provides suggestions for programs to optimally interpret standardized letters of recommendation and minimize bias when writing letters of recommendation. Methods: Available standardized letters of recommendation were those from interviewed applicants to the integrated plastic surgery program between 2015 and 2018 (82 total applicants) and all applicants in the year of 2019 (241 total applicants). Various standardized letters of recommendation component scores were analyzed by demographics of both applicants and letter writers, including descriptive statistics, analysis of variance, t testing, and Fleiss kappa for interrater reliability using SPSS software (IBM, Armonk, N.Y.). Results: No scoring category had an average score below the eighty-fifth percentile. Male letter writers scored male applicants significantly higher in Overall, Conscientiousness, Self-Initiative, and Academic Performance components, whereas female letter writers scored female applicants higher in Communication and Overall components. In the 2019 cohort, applicants of a minority race received lower scores on average in nine out of 10 categories, a difference that was significant in the Team Player category. Conclusions: Score inflation pervades standardized letters of recommendation scoring in plastic surgery. Work must be done by letter writers to help mitigate racial and gender biases that influence recommendations for resident selection in plastic surgery.

Deconstructing the Excellent Plastic Surgeon: A Survey of Key Attributes

“…19,22,32 However, even the most well-written letter does not always offer insight into these qualities, and it is well known that implicit biases seep through into the content of letters of recommendation, particularly when it comes to description of these soft skills. 33 Additionally, recommendation letters written by well-connected plastic surgeons are preferentially valued, introducing another source of bias when evaluating soft skills from narrative letter content. 34,35 The results of our survey suggest that we should strive to find ways to evaluate, and more importantly, prioritize these traits in the application review process.…”

“…However, a recent publication by Reghunathan et al revealed that this question format is associated with significant score inflation and has therefore become dubiously useful in differentiating applicants. 33 Furthermore, both score inflation and biases against demographic minorities persist even in standardized letters of recommendation, with applicants of minority races receiving lower scores on average when compared with their peers. 33 To address this challenge of distinguishing candidates from a highly qualified applicant pool, holistic review is critical.…”

“…33 Furthermore, both score inflation and biases against demographic minorities persist even in standardized letters of recommendation, with applicants of minority races receiving lower scores on average when compared with their peers. 33 To address this challenge of distinguishing candidates from a highly qualified applicant pool, holistic review is critical. Holistic review, described by the AAMC as "missionaligned admissions or selection processes" that consider the "whole" applicant, requires that programs identify their values and focus resident selection on relevant attributes and experiences alongside metrics.…”

Explicit Values as Guides to Recruitment, Program Identity, and Culture

“…Letters of recommendation can convey bias. 2 Aside from this, letters of recommendation require a degree of interpretation. A strong letter is not the longest letter, recounting the accomplishments already noted on the curriculum vitae.…”

“PREPPED: Plastic Surgery Research, Education, and Preparation Promoting Equity and Diversity”

“…Inequities observed in the physician workforce result from multiple factors: bias from evaluators, socioeconomic disadvantage, unequal access to resources, and inadequate availability of mentorship, among others. 4,5 PREPPED is a concerted effort by several invested faculty and residents to assist a cohort of capable, highly motivated historically disadvantaged students interested in the field of plastic surgery. PREPPED represents the first national course for subinternship preparation for underrepresented students in plastic surgery (Figs.…”