The Association Between USMLE Step 2 Clinical Knowledge Scores and Residency Performance: A Systematic Review and Meta-Analysis

Abstract

Purpose With the change in Step 1 score reporting, Step 2 Clinical Knowledge (CK) may become a pivotal factor in resident selection. This systematic review and meta-analysis seeks to synthesize existing observational studies that assess the relationship between Step 2 CK scores and measures of resident performance. Method The authors searched MEDLINE, Web of Science, and Scopus databases using terms related to Step 2 CK in 2021. Two researchers identified studies investigating the association between Step 2 CK and measures of resident performance and included studies if they contained a bivariate analysis examining Step 2 CK scores’ association with an outcome of interest: in-training examination (ITE) scores, board certification examination scores, select Accreditation Council for Graduate Medical Education core competency assessments, overall resident performance evaluations, or other subjective measures of performance. For outcomes that were investigated by 3 or more studies, pooled effect sizes were estimated with random-effects models. Results Among 1,355 potential studies, 68 met inclusion criteria and 43 were able to be pooled. There was a moderate positive correlation between Step 2 CK and ITE scores (0.52, 95% CI 0.45–0.59, P < .01). There was a moderate positive correlation between Step 2 CK and ITE scores for both nonsurgical (0.59, 95% CI 0.51–0.66, P < .01) and surgical specialties (0.41, 95% CI 0.33–0.48, P < .01). There was a very weak positive correlation between Step 2 CK scores and subjective measures of resident performance (0.19, 95% CI 0.13–0.25, P < .01). Conclusions This study found Step 2 CK scores have a statistically significant moderate positive association with future examination scores and a statistically significant weak positive correlation with subjective measures of resident performance. These findings are increasingly relevant as Step 2 CK scores will likely become more important in resident selection.

Reducing implicit bias in the neurosurgery application and interview process: a single-institution experience

“…Noteworthy studies have demonstrated the presence of racial biases associated with USMLE board scores and AΩA status. 3,8,10,11,[16][17][18] Our primary aim encompassed the intentional mitigation of established racial biases during the initial phases of the selection process, with the intention of preempting their impact on our interview evaluations, which we contend hold greater importance in shaping our rank list. It is pertinent to acknowledge that our application process remains an ongoing and dynamic endeavor, characterized by a holistic approach that continues to evolve.…”

“…Although Shirkhodaie et al showed that Step 2 CK scores have a statistically significant moderate positive association with future examination scores, they found a statistically significant weak positive correlation with overall, subjective measures of resident performance. 8 However, Udawatta et al demonstrated no significant correlation between USMLE Step 2 CK scores and ABNS written examination scores. 9 AΩA status has been shown to not correlate positively with faculty evaluations, in-training examinations, board examinations, chief resident selection, or research productivity across a variety of specialties.…”

Ophthalmology Residency Program Director Survey on Pass/Fail U.S. Medical Licensing Exam Step 1 Scoring

“…7 However, even with USMLE Step 2 exams, the correlation between scores and residency performance remains uncertain. 8…”

“…7 However, even with USMLE Step 2 exams, the correlation between scores and residency performance remains uncertain. 8 To address these concerns, the Federation of State Medical Boards (FSMB) and the NBME partnered with the American Medical Association, the Association of American Medical Colleges, and the Educational Commission for Foreign Medical Graduates to convene a summit, the Invitational Conference on USMLE Scoring. Following discussions at the summit and feedback solicited on a website, the FSMB and NBME made the decision to transition to P/F scoring.…”