Correlations Between the USMLE Step Examinations, American College of Physicians In-Training Examination, and ABIM Internal Medicine Certification Examination

Abstract

Purpose To assess the correlations between United States Medical Licensing Examination (USMLE) performance, American College of Physicians Internal Medicine In-Training Examination (IM-ITE) performance, American Board of Internal Medicine Internal Medicine Certification Exam (IM-CE) performance, and other medical knowledge and demographic variables. Method The study included 9,676 postgraduate year (PGY)-1, 11,424 PGY-2, and 10,239 PGY-3 internal medicine (IM) residents from any Accreditation Council for Graduate Medical Education–accredited IM residency program who took the IM-ITE (2014 or 2015) and the IM-CE (2015–2018). USMLE scores, IM-ITE percent correct scores, and IM-CE scores were analyzed using multiple linear regression, and IM-CE pass/fail status was analyzed using multiple logistic regression, controlling for USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores; averaged medical knowledge milestones; age at IM-ITE; gender; and medical school location (United States or Canada vs international). Results All variables were significant predictors of passing the IM-CE with IM-ITE scores having the strongest association and USMLE Step scores being the next strongest predictors. Prediction curves for the probability of passing the IM-CE based solely on IM-ITE score for each PGY show that residents must score higher on the IM-ITE with each subsequent administration to maintain the same estimated probability of passing the IM-CE. Conclusions The findings from this study should support residents and program directors in their efforts to more precisely identify and evaluate knowledge gaps for both personal learning and program improvement. While no individual USMLE Step score was as strongly predictive of IM-CE score as IM-ITE score, the combined relative contribution of all 3 USMLE Step scores was of a magnitude similar to that of IM-ITE score.

Novel Formative Approach of the ESAP-ITE Provides Strong Predictive Value for ABIM Certification Outcomes

“…These findings are consistent with reports from other internal medicine subspecialties showing that ITEs and prior standardized testing performance also predict outcomes on their respective subspecialty certification examinations [ 4-9 ]. Younger age on date of ABIM-ECE was the only other significant predictor, and this result also mirrors prior studies that found younger test-takers were more likely to pass initial board certification examinations on the first attempt [ 3 , 7 , 22 ]. Although reasons for this are speculative, Jeffe and Andriole have suggested that such results indicate older graduates may experience greater difficulties in timely advancement toward board certification [ 22 ].…”

“…When used as intended, the ESAP-ITE represents an important resource for assessment of medical knowledge and self-identification of potential knowledge gaps. McDonald et al have shown that “a systematic, frequent (eg, daily), modest (eg, ≤20 minutes per day), intentional approach to medical knowledge acquisition” is associated with significant gains in knowledge over time [ 3 , 23 ]. Such an approach, when guided by ESAP-ITE results, could conceivably increase odds of an “at-risk” test-taker passing the ABIM-ECE.…”

“…Results from this annual evaluation are used by program directors to focus educational topics in subsequent years, as well as to monitor trainees’ progress toward competency and preparedness to sit for the board certification examination at the completion of training. Indeed, it is well-recognized that performance on the Internal Medicine ITE provides some predictive value for trainee success when attempting board certification [ 2 , 3 ].…”

Curriculum Transition From Lecture-Based to Team-Based Learning is Associated With Improved Performance on Internal Medicine In-Training Examination

“…The IM-ITE has been demonstrated to correlate with rate and degree of knowledge acquisition during residency 13 and predicts performance on the American Board of Internal Medicine Certification Examination (IM-CE). [14][15][16] Furthermore, higher performance on the IM-CE has been associated with better patient care outcomes. [17][18][19] Studies have demonstrated that TBL in undergraduate medical education can lead to improvements in students' subject area standardized testing performance.…”

The use of the NEJM knowledge + online platform to supplement traditional pulmonary didactic: a resident-led educational quality improvement project at a community hospital IM GME program

“…Once the Accreditation Council of Graduate Medicine Education’s Next Accreditation System (ACGME-NAS) was implemented in July 2014, a new continuing accreditation metric requiring a program to achieve and maintain a recent, rolling 3-year first attempt pass rate of >80% on the ABIM-CE by their graduates[ 1 ] likely led to a proliferation and increased use of such online tools. The IM-ITE performance has been correlated with passing the ABIM-CE and such supplemental use for remediation has successfully been reported [ 2 ].…”