The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study

Abstract

Background The objective structured clinical examination (OSCE) is a part of emergency medicine (EM) examinations such as the Masters of Medicine in Emergency Medicine (MMed) examination and the equivalent Member of the Royal College of Emergency Medicine (MRCEM) examination. The use of formative OSCEs to prepare EM residents for summative OSCEs has not been investigated. This study aimed to evaluate the role of formative OSCEs in preparing EM residents for the MMed and MRCEM OSCE. Methods This was an observational, retrospective, mixed-methods cohort study. We analysed data from formative OSCEs conducted by the National Healthcare Group EM residency programme from 2013 to 2019, and from a questionnaire distributed to all residents during the study period. Residents’ formative OSCE participation and scores were compared with first-attempt summative OSCE success. Qualitative analysis of residents’ opinions on the usefulness of the formative OSCE was performed. Results Forty-three of the 50 (86.0%) residents attended at least one formative OSCE. Of the 46 who responded to the questionnaire, 40 (87.0%) had attempted and succeeded in the MMed or MRCEM OSCE, of whom 35 (87.5%) had succeeded on the first attempt. Residents who succeeded in the summative OSCE on the first attempt tended to have higher proximate (mean = 70.6, SD = 8.9 vs mean = 64.3, SD = 10.8) and mean (mean = 67.4, SD = 7.1 vs mean = 62.8, SD = 7.3) formative OSCE scores. All 8/40 (20.0%) residents who attended more than three formative OSCEs succeeded in the summative OSCE on their first attempt. Residents’ formative OSCE scores tended to improve with successive formative OSCEs, demonstrating a positive training effect. All residents felt that the formative OSCE was useful in preparing them for the summative OSCE. Conclusions Participation in multiple formative OSCEs was beneficial in preparing residents for the summative OSCE. The formative OSCE was useful in familiarising residents with the examination, giving them an opportunity to perform in near-examination conditions, and providing feedback to residents and faculty about their progress. Our findings may support the implementation of formative OSCEs in other training programmes to prepare learners for high-stake summative OSCEs.

Objective structured clinical examination versus traditional written examinations: a prospective observational study

“…Showing a progression between the two OSCE sessions in naïve students demonstrated the impact of training in real conditions. These results are frequently found in studies evaluating the skills of students or doctors in the context of medical simulation [ 18 , 24 , 25 ]. OSCE scores were found to have high reliability and demonstrated significant differences in performance by year of training, providing evidence for the validity of OSCE scores as markers of progress in learners at different levels of training [ 26 ].…”

Impact of Formative Objective Structured Clinical Examination (OSCE) on Clinical Performance of Undergraduate Medical Students in Summative OSCE

“…(20) Lee et al discovered similar results in a mixed cohort study of emergency medicine residents. (21) In a 2019 study conducted in the United States, Sampat et al discovered that students who completed a practice exam performed much higher in their end-of-clinic OSCE. (22) Another study conducted in 2017 at the Caribbean Medical School found a significant improvement in academic performance following the use of formative assessment in students in the basic science years.…”

Stress and value: the student perspective on utilizing real vs. actor patients in objective structured clinical examinations

“…Since Dr. Ronald Harden’s inception of the objective structured clinical examination (OSCE) in the 1970’s the OSCE has spread around the globe and cemented itself as an educational pillar spanning all levels of training and a wide range of health professions [ 1 – 6 ]. Today, OSCE’s serve both summative and formative functions [ 7 – 9 ]. They offer the ability to have nearly identical encounters which increased the possibility of reliable, valid and replicable assessments with definable goals for performance which allows for the use of OSCEs as a summative assessment of a trainee.…”