The Effects of Reusing Written Test Items: A Study Using the Rasch Model

Abstract

Background. Relevant literature reports no increase in individual scores when test items are reused, but information on change in item difficulty is lacking. Purpose. To test an approach for quantifying the effects of reusing items on item difficulty. Methods. A total of 671 students sat a newly introduced exam in four testing shifts. The test forms experimentally combined published, unused, and reused items. Figures quantifying reuse effects were obtained using the Rasch model to compare item difficulties from different person samples. Results. The observed decrease in mean item difficulty for reused items was insignificant. Students who self-scheduled to the last test performed worse than other students did. Conclusion. Availability of leaked material did not translate into higher individual scores, as mastering leaked material does not guarantee transfer of knowledge to new exam items. Exam quality will not automatically deteriorate when a low ratio of randomly selected items is reused.

Analyzing the decline of student scores over time in self‐scheduled asynchronous exams

“…During Spring 2009, Wagner‐Menghin, Preusche, and Schmidts () conducted an asynchronous paper‐and‐pencil exam consisting of only multiple‐choice questions. A total of 671 students were initially assigned to four time slots and allowed to reschedule their exam times as they wished.…”

“…A total of 671 students were initially assigned to four time slots and allowed to reschedule their exam times as they wished. By comparing the difficulty of reused items using the Rasch Model, Wagner‐Menghin et al () observed that reused items became easier after their first use.…”

“…This asynchronous exam has one of the more negative slopes in our data set, and we chose it here because the highly negative slope is easy to discern [Color figure can be viewed at wileyonlinelibrary.com] allowed to reschedule their exam times as they wished. By comparing the difficulty of reused items using the Rasch Model, Wagner-Menghin et al (2013) observed that reused items became easier after their first use. The relationship between procrastination and academic performance has been extensively studied (see Kim & Seo, 2015, and studies cited therein).…”

Re-using questions in classroom-based assessment: An exploratory study at the undergraduate medical education level

“…The development of assessment that appropriately and purposefully assesses students’ knowledge and skills, provides feedback, and that yields reliable scores, challenges even the best educators [ 5 – 7 ]. Even more so when one has to limit effects of potential content leakage and thus has to construct new items yearly [ 8 ]. The observed increase in assessment frequency is undoubtedly an important additional strain on medical education program administrators and faculty members who may already struggle to develop quality assessment.…”

“…In one study on the topic of repeated questions’ psychometric properties, Wagner et al. [ 8 ] concluded that there was little impact of question exposure in the context of a formative assessment, however they lacked the power to observe a statistical difference.…”