Perspectives of Orthopedic Surgery Program Directors on the USMLE Step 1 Scoring Change

Abstract

The orthopedic surgery residency application process is highly competitive, and residency programs rely on objective measures, such as the United States Medical Licensing Examination (USMLE) Step 1 score, to assist in applicant selection. To deemphasize standardized test scores and improve student well-being, governing bodies have elected to change the Step 1 examination to a pass/fail grading system beginning in 2022. Given the utility of Step 1 in the orthopedic surgery residency application process, this change will significantly affect how applicants are assessed. To determine how this change will affect the process, we developed and validated a 19-item anonymous electronic survey and invited program directors (PDs) from orthopedic surgery residency programs accredited by the Accreditation Council for Graduate Medical Education to respond. The 86 surveys that were completed represented a 43.2% response rate (86 of 199). Only 4.7% of orthopedic surgery PDs believe that this change is a good idea, and 91.7% of PDs believe that the change will make it more difficult to compare applicants objectively. In addition, 90.7% of PDs report that they will increase emphasis on the Step 2 CK (clinical knowledge) examination in comparing residency applicants for their program, with 90.4% of PDs reporting that they will require students to submit their Step 2 scores via the Electronic Residency Application Service. These results indicate that changing the Step 1 scoring to pass/fail is unpopular among orthopedic surgery PDs. Further, losing the numeric Step 1 score will increase reliance on Step 2 scores and emphasis on less objective measures, such as where an applicant attended medical school. [ Orthopedics . 2022;45(5):e257–e262.]

Residency Program Directors’ Views on Research Conducted During Medical School: A National Survey

“…The observation that some PDs will consider research of equal or greater importance than the Step 2 score is provocative. The changing significance of Step 2 is a topic that has been hotly debated of late, with multiple survey studies reporting and blog posts opining that PDs will consider Step 2 “the new Step 1.” 46–57 The sentiment that Step 2 will be relied on more heavily, at least compared with student research activity, was not expressed by respondents in this study. Most likely, the process of selecting applicants for interviews and ranking without a numeric Step 1 score will take several years to refine, and the question of whether research continues to be of equal or greater importance to the Step 2 score will merit revisiting at a future date.…”

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis

“…After the initial search, 351 duplicates were removed and 44 were excluded after title and abstract review, 4,[7][8][9][10][11][12][13][14][15] based on inclusion and exclusion criteria. [16][17][18][19][20][21][22][23][24][25] Full-text review was performed on 98 articles, [26][27][28][29][30][31][32][33][34][35] of which 45 studies were included [36][37][38][39][40][41][42][43][44][45][46][47][48][49][50] (see Figure, Supplemental Digital Content 5, which illustrates a flow-chart of our literature search, as well as our full search strategies).…”

“…Twenty-eight studies [7][8][9][10][11][12][13][14][15][16][17][19][20][21][22][23][24][26][27][28][29]32,33,35,41,44,46,48 included in the systematic review examined disagreement among PDs with the Step 1 score change, of which 12 studies 7,9,11,12,[19][20][21][22]24,28,29,41 1).…”

“…Eleven studies 7,9,10,[19][20][21]24,28,29,32,41 included in the systematic review examined perceptions among PDs regarding the arduousness of the applicant screening process due to the score change; all studies qualified for the meta-analysis ( I 2 = 67.9%) of all PDs agreed that the applicant screening process will become more arduous, with high heterogeneity and low publication bias (see 2, Figure 1), with high heterogeneity among both groups.…”

CORR® Curriculum—Orthopaedic Education: Will Holistic Review of Residency Candidates be as Painful as You Think?

“…In 2022, The National Board of Medical Examiners (NBME) and Federation of State Medical Boards (FSMB) changed the Step 1 exam to pass/fail, in part to encourage more holistic review of applicants [4]. This grading change will likely just shift focus from Step 1 to Step 2 Clinical Knowledge (CK) scores for many residencies [3,8]; after all, this simple change is far less daunting than transforming a program's selection process into a holistic evaluation.…”