Introduction: Competency assessment has a substantial impact on both the quality of education and the quality of care. Debre Tabor University has been administering pre-internship qualification exams to ensure clinical competence and readiness for internship practice. A variety of assessment methods, including MCQs, objective structured clinical exams, and external exams were used. The reliability of results can be determined by how consistently students perform across different assessment formats. Objective: The objective of this study was to compare the scores obtained on the different assessment formats used in the pre-internship qualification examination. Methods: A correlational study was conducted at the College of Health Sciences, Debre Tabor University, Debre Tabor, Ethiopia from November to December 2019. Participants of the study were medical students who completed their fifth-year medical education. All 46 fifth-year medical students who took the qualifying exam were included in the analysis. The data were analyzed using descriptive statistics such as percentage and mean as well as inferential statistics like Pearson's correlation. The analysis was done with STATA software. Results: The mean performance score ranged from 67.9 on multiple-choice questions to 77.2 on viva. Relatively, students performed better on the viva test and poorly on the OSCE. There was no significant difference in the overall performance of male and female students, however, female students were better in long- and short-case exams. For female students, the correlation between performance scores was stronger. When compared to their written exam results, female students performed much better on the long-case, short-case, and oral exams, but men did not show such a difference. Conclusions: Students performed better on the oral examination. The study demonstrated a disparity between female and male students on the level of performance in the clinical skills exams. Among female students, there was a substantial correlation between cognitive ability and clinical performance. Gender was an effect modifier for the correlation between cognitive ability and clinical skill performance in the medical qualification examination.