Evaluating Functional and Non-Functional Distractors and Their Relationship with Difficulty and Discrimination Indices in Four-Option Multiple-Choice Questions

Abstract

Multiple-choice questions-one best answer (MCQ-OBA) is the most frequently accepted assessment tool in Iran’s medical universities. Writing functional distractors (FDs) is an important aspect of framing MCQ-OBA. This study aimed to assess FDs and non-functional distractors (NFDs) in MCQ-OBA and the association of distractor efficiency with difficulty and discrimination indices. This cross-sectional study was conducted at the Department of Immunology, the Ahvaz Jundishapur University of Medical Sciences. A total of 734 MCQ-OBA were reviewed, including 2,936 options (2,202 distractors and 734 correct responses). NFDs were defined as options chosen by <5% of examinees. Of the 734 MCQ-OBAs, 265 (36.1%) had 0 NFDs, 231 (31.5%) had 1 NFD, 146 (19.9%) had 2 NFDs, and 92 (12.5%) had 3 NFDs. The Pearson’s correlation showed a significant relationship between the difficulty index and the number of NFDs (r = 0.453; P < 0.0001). However, the correlation between the discrimination index and the number of NFDs was insignificant (r = 0.055; P = 0.135). The findings revealed that NFDs inversely affected the test quality of items. Items with more NFDs were easier and had poorer discriminatory power. The distractor function analysis and revision of NFDs serve as an important method to improve the quality of MCQ-OBA.

Analisis Kualitas Tes dan Butir Soal Integral pada Evaluasi Formatif Matematika Teknik

“…( 4) If DI ≤ 0,19, then the item should be eliminated or completely revised. Lebih lanjut (Botterman et al, 2022;Christian et al, 2017;Date et al, 2019;Kelley, 1939;Kumar Namdeo & Dev Rout, 2016;Masters, 1996;Shakurnia et al, 2022) mengklasifikasikan DI dalam 4 kriteria antara lain: DI ≤ 0,20 (poor), DI 0,21-0,24 (acceptable), DI 0, 25-0,34 (good) and DI ≥ 0,35 (excellent).…”

“…Menulis Functioning Distractors (FD) adalah aspek penting dalam menyusun pertanyaan pilihan ganda yang berkualitas. Butir soal pilihan ganda dengan distraktor yang efektif sangat penting untuk memperoleh tes yang valid (Shakurnia et al, 2022).…”

“…Penelitian yang sejalan dilakukan oleh (Kumar et al, 2021) yang melaporkan bahwa 73% FD dan 27% NFD. Penelitian (Date et al, 2019;Quaigrain & Arhin, 2017;Rao et al, 2016;Sajjad et al, 2020;Shakurnia et al, 2022) yang melaporkan bahwa 70%; 63,71%; 89,99%; 74,7%; dan 63,9% item adalah FD, 30%; 33,29%; 10,01%; 25,3%; dan 31,6% item adalah NFD.…”

“…Saat menyusun soal pilihan ganda yang berkualitas baik, aturan utamanya adalah bahwa pengecoh harus masuk akal, yaitu ditempatkan dekat dengan jawaban yang benar yang akan meningkatkan kemungkinan memilih pengecoh ini dibandingkan jawaban yang benar oleh mahasiswa (Date et al, 2019). Sejalan dengan hasil penelitian (Shakurnia et al, 2022) yang menyimpulkan bahwa NFD berbanding terbalik dengan kualitas tes. Namun, soal dengan FD yang lebih banyak akan lebih sulit dan memiliki kekuatan diskriminatif yang lebih besar.…”

The Role of Qualitative and Quantitative Feedback on Faculties’ Quality of Writing Multiple Choice Questions

“…A total of 1525 distractor options (62.9%) were functional distractors (FD), and 889 (37%) were non-functional distractors (NFDs). The results showed that the MCQs should be improved (5). Meanwhile, the analysis of the questions of the specialized midwifery courses of the same university was desirable (3).…”