Impact of familiarity with the format of the exam on performance in the OSCE of 5th undergraduate medical students – an interventional study

Abstract

Background: Assessments aim to verify whether students have acquired the necessary competencies. These assessments are typically carried out as summative structured examinations. It is important not only to establish a constructive alignment between teaching and assessment but also to familiarize students with the examination format prior to the assessment. The latter point is intended to ensure that true competency is measured. Regarding practical examinations such as objective structured clinical examinations (OSCEs), this is often not the case. Therefore, it is unclear whether students can demonstrate their true potential or possibly perform less effectively due to the unfamiliar examination format. Hence, we questioned whether a 10-minute active familiarization in the form of simulation improved medical students´ OSCE performance. Next, we wanted to elucidate whether the effect depends on whether the familiarization procedure is active or passive. Methods: We implemented an intervention consisting of a 10-minute active simulation to prepare the students for the OSCE setting. We compared the impact of this intervention on performance to no intervention in 5th-year medical students (n=1284) from 2018 until 2022. Recently, a passive lecture, in which the OSCE setting is explained without active participation of the students, was introduced as a comparator group. Students who participated in neither the intervention nor the passive lecture group formed the control group. The OSCE performance between the groups and the impact of gender was assessed using X2, nonparametric tests and regression analysis (total n=362). Results: We found that active familiarization of students in the exam setting yields significantly better performance compared to the passive comparator (Cohen´s d=0.857, p<0.001)) and control group (Cohen´s d=0.473, p<0.001). In multivariate regression analysis, active intervention remained the only significant variable with a 2.945-fold increase in the probability of passing the exam (p=0.018). Conclusions: A short 10-minute active intervention to familiarize students with the OSCE setting significantly improved student performance. We suggest that curricula should include simulations on the exam setting in addition to courses that increase knowledge or skills to mitigate the negative effect of nonfamiliarity with the OSCE exam setting on the student.