Background The primary goal of residency programs is to select and educate qualified candidates to become competent physicians. Program directors often use performance on licensure examinations to evaluate the ability of candidates during the resident application process. The American College of Osteopathic Family Physicians (ACOFP) administers an in-service examination (ISE) to residents annually. There are few prior studies of the relationship between the Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) series and formative assessments of residents in training.Objective We explored the relationship between performance on COMLEX-USA and the ACOFP in-service examination to offer support on the use of licensing examinations in resident selection.Methods In 2016, performance data from the COMLEX-USA and the ISE were matched for 3 resident cohorts (2011-2013, inclusive; N ¼ 1384). Correlations were calculated to examine the relationship between COMLEX-USA and ISE scores. Multiple linear regression models were used to determine if performance on COMLEX-USA significantly predicted third-year ISE (ISE-3) scores.