Use of very short answer questions compared to multiple choice questions in undergraduate medical students: An external validation study

Abstract

Multiple choice questions (MCQs) offer high reliability and easy machine-marking, but allow for cueing and stimulate recognition-based learning. Very short answer questions (VSAQs), which are open-ended questions requiring a very short answer, may circumvent these limitations. Although VSAQ use in medical assessment increases, almost all research on reliability and validity of VSAQs in medical education has been performed by a single research group with extensive experience in the development of VSAQs. Therefore, we aimed to validate previous findings about VSAQ reliability, discrimination, and acceptability in undergraduate medical students and teachers with limited experience in VSAQs development. To validate the results presented in previous studies, we partially replicated a previous study and extended results on student experiences. Dutch undergraduate medical students (n = 375) were randomized to VSAQs first and MCQs second or vice versa in a formative exam in two courses, to determine reliability, discrimination, and cueing. Acceptability for teachers (i.e., VSAQ review time) was determined in the summative exam. Reliability (Cronbach’s α) was 0.74 for VSAQs and 0.57 for MCQs in one course. In the other course, Cronbach’s α was 0.87 for VSAQs and 0.83 for MCQs. Discrimination (average Rir) was 0.27 vs. 0.17 and 0.43 vs. 0.39 for VSAQs vs. MCQs, respectively. Reviewing time of one VSAQ for the entire student cohort was ±2 minutes on average. Positive cueing occurred more in MCQs than in VSAQs (20% vs. 4% and 20.8% vs. 8.3% of questions per person in both courses). This study validates the positive results regarding VSAQs reliability, discrimination, and acceptability in undergraduate medical students. Furthermore, we demonstrate that VSAQ use is reliable among teachers with limited experience in writing and marking VSAQs. The short learning curve for teachers, favourable marking time and applicability regardless of the topic suggest that VSAQs might also be valuable beyond medical assessment.

Does the Proportion of Marks for Wet Laboratories Affect the Overall Mark, Grade, and Failure Rates?

“…One possible reason for this is that the final exam contained SAQs in addition to MCQs, whereas the mid-semester exam was solely MCQs. A recent study has shown that medical students perform better in MCQs than SAQs in examinations (van Wijk et al, 2023). As a follow-on, the difference between the marks for the exams and wet laboratories in the present study could have been even greater if SAQs had been included in the mid-semester exam.…”

Describing the landscape of medical education preprints on medRxiv

“…From preprint to published article, 7 articles (7.3%) adjusted author order, but retained the same authorship team. Of these articles, only one changed the first author 35 and in two cases the last author was shifted to a new author order location. 36,37 Thirteen preprints (13.5%) had authors added (n=10) or removed (n=3) when they later appeared in a peer-reviewed journal.…”