A Validity Study of COMLEX-USA Level 2-CE and COMAT Clinical Subjects: Concurrent and Predictive Evidence

Abstract

Background The Comprehensive Osteopathic Medical Licensure Examination (COMLEX-USA) Level 2–Cognitive Examination (CE) and the Comprehensive Osteopathic Medical Achievement Test (COMAT) are administered to similar populations (third- and fourth-year osteopathic students) at similar points in time. Examining the relationship between scores on the 2 assessments that measure similar constructs ultimately supports the validity of both. Objective The purpose of this study is to provide empirical evidence of the concurrent and predictive validity of COMAT and COMLEX-USA Level 2-CE. Methods In 2018, first-attempt scores on Level 2-CE were aggregated from June 2015 to May 2018 and matched with first-attempt scores on each COMAT clinical subject. We conducted correlational analyses between performance on COMAT and Level 2-CE, and COMAT scores and Level 2-CE discipline subscores. Additionally, we used multivariate regression to analyze the predictive relationship between performance on all COMAT clinical subjects and Level 2-CE. Results The results from correlational analyses indicated statistically significant, positive associations between COMAT and Level 2-CE scores (r = 0.49–0.68, P < .0001), and statistically significant, but slightly weaker relationships between COMAT scores and Level 2-CE discipline subscores (r = 0.31–0.60, P < .0001). Furthermore, results from the multiple regression indicated that scores on COMAT explained 68% of the variance in Level 2-CE scores, and that COMAT internal medicine and emergency medicine were weighted more heavily than other specialties. Conclusions The findings from this study can inform assessment practices by supporting the use of COMAT for osteopathic medical schools that do not administer COMAT.

Effects of Organ System Courses of the First Two Years of Medical School on the Performance of the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 2-Cognitive Evaluation

“…This is because PCM III primarily teaches internal medicine, emphasizing the diagnosis and treatment of diseases. It parallels the finding that the National Board of Osteopathic Medical Examiners (NBOME) COMAT Internal Medicine and Emergency Medicine performances in the third-year rotation are the most significant predictors for COMLEX Level 2-CE because Level 2-CE emphasizes primary care as a generalist examination [10,11]. If students perform poorly in PCM III, they may need additional assistance when preparing for Level 2-CE.…”

“…They found that the highest correlation with the COMLEX Level 2-CE score was the COMLEX Level 1 score, followed by the performance in the first two years of medical school, total grade point average (GPA) as a fourth-year student, and clinical GPA [9]. Furthermore, since the Comprehensive Osteopathic Medical Achievement Test (COMAT) series of subject examinations was installed in 2011, two studies indicated that scores on COMAT explained 63-68% of the variance in Level 2-CE scores, with COMAT Internal Medicine and Emergency Medicine displaying the highest correlations [10,11]. Using the COMAT performance to predict COMLEX Level 2-CE is too late for students since they will not have enough time to modify and improve their preparation for COMLEX Level 2-CE.…”

Establishing the Validity of Licensing Examination Scores

“…In this issue of the Journal of Graduate Medical Education, Hudson and colleagues provide data indicating that performance on the Comprehensive Osteopathic Medical Licensure Examination (COM-LEX-USA) Level 2-Cognitive Examination (CE) is related to performance on the Comprehensive Osteopathic Medical Achievement Test (COMAT). 6 This finding adds to the existing literature documenting the associations between COMLEX performance and both school-based and residency performance measures. [7][8][9] Approximately 24% to 46% of the variance of Level 2-CE scores can be explained by the COMAT scores.…”

Factors Leading to Successful Performance on U.S. National Licensure Exams for Medical Students: A Scoping Review

“…Notably, the large, multisite studies that we included consistently estimated that 90% of NLE performance was explained by individual characteristics rather than school-level variables. [76][77][78][79] Similarly, individuals' performance on curricular assessments was associated with NLE scores, including in the largest studies we found, [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91] while curriculum types or structures were not. 79,86,90,92,93 As schools continue to be concerned about admitting students who will progress without the need for remediation and as they continue to refine models that predict academic achievement, 39,40,94 they should note that the evidence suggests that past academic performance remains a robust predictor of future academic performance.…”