Analysis of multiple choice questions from a formative assessment of medical students of a medical college in Delhi, India

Abstract

Background: Assessment is a dominant motivator to direct and drive students learning. Different methods of assessment are used to assess medical knowledge in undergraduate medical education. Multiple choice questions (MCQs) are being used increasingly due to their higher reliability, validity, and ease of scoring. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), and distracter efficiency (DE).Methods: Students of second year MBBS appeared in a formative assessment test, that was comprised of 50 “One best response type” MCQs of 50 marks without negative marking. All MCQs were having single stem with four options including, one being correct answer and other three incorrect alternatives (distracter). Three question paper sets were prepared by disorganizing sequence of questions. One of the three paper sets was given to each student to avoid copying from neighboring students. Total 50 MCQs and 150 distracters were analyzed and indices like DIF I, DI, and DE were calculated.Results: Total Score of 87 students ranged from 17 to 48 (out of total 50). Mean for difficulty index (DIF I) (%) was 71.6+19.4. 28% MCQs were average and “recommended” (DIF I 30-70%). Mean for discrimination index (DI) was 0.3+0.17. 16% MCQs were “good” and 50% MCQs were in “excellent” criteria, while rests of the MCQs were “discard/poor” according to DI criteria. Mean for distracter efficiency (DE) (%) was 63.4+33.3. 90% of the items were having DE from 100 to 33%. It was found that MCQs with lower difficulty index (<70) were having higher distracter efficiency (93.8% vs. 6.2%, p=0.004).Conclusions: Item analysis provided necessary data for improvement in question formulation and helped in revising and improving the quality of items and test also. Questions having lower difficulty index (<70) were significantly associated with higher discrimination index (>0.15) and higher distractor efficiency.

Item Analysis of Multiple-choice Questions (MCQs): Assessment Tool For Quality Assurance Measures

“…Range of scores/performances as moderately difficult (DI: 0.4-0.8) and sufficient discrimination point biserial correlation (RPB) more or equal to 0.2. It can also increase the standard deviation and the variance of the scores [23][24][25]. For the assumption that any test can contain score error, SEM is used to estimate the interval within which the true score will be obtained.…”

“…The DIF-I can range from zero to a hundred percent. If it is >70%, it is an easy item; 30-70% means average/acceptable difficulty; <30% means a difficult item [18][19][20][21][22][23]. If an MCQ includes 0-NFD, 1-NFD, 2-NFD, or 3-NFD, it means that its capability to act as an efficient distractor is 100, 66.6, 33.3, or 0%, respectively [12][13][14][15][16].…”

“…NFDs include options other than the right answer chosen by <5% of the examinees. Implausible distractors can be noticed easily, so they ought to be modified or rejected [18][19][20][21][22][23].…”

Çoktan Seçmeli Sınavlarda Kullanılan Ölçme ve Değerlendirme Sisteminin Uygulanması: Madde Analiz Örneği

“…Öğrenciler başarı durumlarına göre üst grup ve alt grup olarak sınıflandırıldı. Öğrencilerin yüksek not alan 1/3 lük kısmı üst grup, düşük not alan öğrencilerin 1/3'ünün alt grubu oluşturduğu kabul edildi (Kaur, 2017;Garg, Kumar, Maria, 2019). Üst grupta bulunan öğrencilerin sayısı 30, alt grupta bulunan öğrencilerin sayısı 26 olarak belirlendi.…”

“…Bir maddeye doğru cevap veren kişi sayısının testi alan tüm kişi sayısına oranı madde güçlüğünü (p) verir. Madde güçlüğü (p) hesaplama formülü aşağıdaki gibidir (Garg, Kumar, Maria, 2019).…”