Predicting pass rates on the american board of internal medicine certifying examination

Abstract

Our objective was to determine the ability of the internal medicine In-Training Examination (ITE) to predict pass or fail outcomes on the American Board of Internal Medicine (ABIM) certifying examination and to develop an externally validated predictive model and a simple equation that can be used by residency directors to provide probability feedback for their residency programs. We collected a study sample of 155 internal medicine residents from the three Virginia internal medicine programs and a validation sample of 64 internal medicine residents from a residency program outside Virginia. Scores from both samples were collected across three class cohorts. The Kolmogorov-Smirnov z test indicated no statistically significant difference between the distribution of scores for the two samples ( z ‫؍‬ R esidents in internal medicine must acquire competency in many content areas during their postgraduate training, including medical knowledge, clinical skills, clinical judgment, interpersonal skills, attitudes, ethics, and humanistic qualities as they apply across a vast range of content areas. 1 To assist both residents and program directors, the American College of Physicians, the Association of Professors of Medicine, and the Association of Program Directors in Internal Medicine developed the first national In-Training Examination (ITE) in internal medicine in 1988. This examination was developed to serve as an assessment tool, primarily for residents at the midpoint of training during the second postgraduate year (PGY2). The ITE enables residents and program directors to receive feedback and identify potential deficiencies before administration of the American Board of Internal Medicine (ABIM) certifying examination.The ABIM certifying examination has been the source of a variety of studies over the last decade. Most of these studies dealt with the ability of the ABIM certifying examination to measure the construct of medical knowledge needed by an internal medicine physician. 1-3 More recently, two studies have dealt specifically with the ability of the ITE to predict outcome on the ABIM certifying examination. 4,5 Both studies demonstrated a significant correlation between the ITE and the ABIM certifying examination scores. However, neither study externally validated its results. Therefore, the purpose of this study was threefold: replicate the previous studies through an alternative mode of analysis (logistic model), develop a predictive model that is validated by external data, and develop a simple equation that can be utilized by residency directors in a spreadsheet format to provide quick and easy ABIM pass/fail probability feedback. METHODSInternal medicine residency directors from the three Virginia medical schools were asked to provide ITE scores obtained during residents ' PGY2 year in 1992' PGY2 year in , 1993' PGY2 year in , and 1994, along with the ABIM pass/fail results for the corresponding classes in 1993, 1994, and 1995 who were taking the examination for the first time. The ITE scores reflect...

Associations Between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) Scores

“…[4][5][6][7][8] Our study, viewed in context of these previous studies, helps to establish the continuum of association between MCAT, USMLE Steps, IM-ITE, and ABIM-CE.…”

“…3 Minimum scores on the IM-ITE are associated with passing scores on the American Board of Internal Medicine Certifying Examination (ABIM-CE). [4][5][6][7][8] The United States Medical Licensing Examination (USMLE) was developed by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME). 9 These well-validated multiple choice examinations are given in three steps designed to assess skills for basic sciences, supervised medical care, and medical care without supervision, respectively.…”

Factors Associated with Medical Knowledge Acquisition During Internal Medicine Residency

“…Several authors have demonstrated the association between IM-ITE score and passing the ABIM-CE. [4][5][6][7] Grossman, et al 5 found that IM-ITE scores above the 35th percentile had a positive predictive value of 89% for passing the ABIM-CE. As the correlation between IM-ITE score and percentile is linear below the 90th percentile with a slope of 3.7, utilizing UpToDate for 20 minutes per day would be associated with a 4.5% score increase, which would result in a percentile increase of 17% for those residents who initially scored below the 90th percentile (see Fig.…”

“…3 This validated exam is predictive of performance on the American Board of Internal Medicine Certification Examination (ABIM-CE). [4][5][6][7] While some studies have attempted to assess the impact of specific learning habits on medical knowledge acquisition, they have demonstrated either no association or very small associations with ITE performance. [8][9][10][11][12][13][14][15][16][17][18][19][20] Importantly, these studies did not account for the influence of prior test performance on subsequent test performance in the analyses of factors associated with medical knowledge acquisition.…”

Pass Rates on the American Board of Family Medicine Certification Exam by Residency Location and Size

“…7 A similar correlation of performance has been found for the Certifying Board Examination and performance of the InTraining Examination in internal medicine. 8 Moreover, medical school training may play a role in performance on the ABFM CE. One study showed that residents educated allopathically performed significantly better than their osteopathic counterparts when mean scaled scores were compared.…”