Standardized letters of recommendation and successful match into otolaryngology

Abstract

Objective Historically, narrative letters of recommendation have been utilized in selection of applicants for Otolaryngology residency programs. In the last two application cycles, our specialty adopted a standardized letter of recommendation (SLOR). The intent was to decrease time burden for letter writers and to provide readers with an “objective” evaluation of applicants. The objective of this study is to determine attributes in the SLOR that correlate with matching into a residency program. Study Design We performed a retrospective study using SLOR, USMLE Step 1 scores and match outcomes of applicants who applied to our institution for the 2013 and 2014 match cycle. Methods We included the following variables from the SLOR in the statistical analysis to determine which ones were associated with matching: patient care, medical knowledge, communication skills, procedural skills, research, initiative and drive, commitment to Otolaryngology, commitment to academic medicine, match potential and USMLE1 scores. Results We identified 532 applicants and 963 SLOR. In successful applicants, scores for patient care, medical knowledge, communication skills, initiative and drive, and match potential were statistically higher (p < 0.05). Scores for professionalism, procedural skills, research, commitment to Otolaryngology, commitment to academic medicine and USMLE Step 1 scores were not higher among successfully matched applicants. Conclusions While SLOR can save time for letter writers and provide an objective description of applicants, the utility of individual domains within the SLOR is questionable. Additionally, it is concerning that applicants’ professionalism and procedural skills are not correlated with matching in our specialty. Level of Evidence NA

Using word clouds to Re‐envision letters of recommendation for residency applicants

“…This lack of variability in ratings ultimately prevents differentiation amongst applicants. Similarly, Kimple et al demonstrated that scores for professionalism, procedural skills, research, commitment to otolaryngology, and commitment to academic medicine were not correlated with successful matching . This further questions the utility of the individual domains within the SLOR.…”

“…Unfortunately, disadvantages of the SLOR became apparent shortly thereafter. These include a lack of variation in ratings, preventing differentiation amongst applicants, and the questionable utility of individual domains within the SLOR . Although the SLOR was arguably developed and implemented to replace the NLOR, the majority of institutions continue to submit both.…”

Fixing letters of “wreck”‐ommendation

“…The academy transitioned to a standardized recommendation form in 2013 and is one of the most important items noted in the candidate's review . In medical residencies, higher standardized recommendation scores were significantly associated with successful matching . However, the overall average recommendation score was 85.6% and 83.3% for matching and non‐matching candidates respectively, highlighting the questionable value of the recommendations …”

“…3,4 In medical residencies, higher standardized recommendation scores were significantly associated with successful matching. 5 However, the overall average recommendation score was 85.6% and 83.3% for matching and non-matching candidates respectively, highlighting the questionable value of the recommendations. 5 Whether due to selection bias, exaggeration, or professional pressure, many letters of recommendation are overtly positive, making it difficult to understand a candidate's abilities and discern qualifications in comparison to other applicants.…”

Objective measures and the standardized letter of recommendation in the otolaryngology residency match

“…However, although the SLOR has been shown to reduce the time burden for letter writers and readers and provide a more objective description of candidates, the utility of the 10 individual domains within the SLOR remains questionable, particularly in a very competitive applicant pool such as otolaryngology, dermatology, and plastic surgery . In this domain, we question whether the SLOR loses sensitivity as a tool when applicants are more uniformly among the academically elite.…”