Comparison in the quality of distractors in three and four options type of multiple choice questions

Abstract

IntroductionThe number of distractors needed for high quality multiple choice questions (MCQs) will be determined by many factors. These include firstly whether English language is their mother tongue or a foreign language; secondly whether the instructors who construct the questions are experts or not; thirdly the time spent on constructing the options is also an important factor. It has been observed by Tarrant et al that more time is often spent on constructing questions than on tailoring sound, reliable, and valid distractors.ObjectivesFirstly, to investigate the effects of reducing the number of options on psychometric properties of the item. Secondly, to determine the frequency of functioning distractors among three or four options in the MCQs examination of the dermatology course in University of Bahri, College of Medicine.Materials and methodsThis is an experimental study which was performed by means of a dermatology exam, MCQs type. Forty MCQs, with one correct answer for each question were constructed. Two sets of this exam paper were prepared: in the first one, four options were given, including one key answer and three distractors. In the second set, one of the three distractors was deleted randomly, and the sequence of the questions was kept in the same order. Any distracter chosen by less than 5% of the students was regarded as non-functioning. Kuder-Richardson Formula 20 (Kr-20) measures the internal consistency and reliability of an examination with an acceptable range 0.8–1.0. Chi square test was used to compare the distractors in the two exams.ResultsA significant difference was observed in discrimination and difficulty indexes for both sets of MCQs. More distractors were non-functional for set one (of four options), but slightly more reliable. The reliability (Kr-20) was slightly higher for set one (of four options). The average marks in option three and four were 34.163 and 33.140, respectively.ConclusionCompared to set 1 (four options), set 2 (of three options) was more discriminating and associated with low difficulty index but its reliability was low.

Item Analysis of Multiple-Choice Questions at the Department of Community Medicine, Wah Medical College, Pakistan

“…Difficulty and discrimination indices have positive association except when the difficulty index is either 2,9 too high or too low. Reliability is also a measurement of internal consistency of assessment and it ranges from 0 to 1, the higher the value the 1,3 more is the reliability. The Optical Machine Reader (OMR) uses software programs to find out the item statistics including p value, point biserial, reliability and standard error.…”

The use of distractors in multiple-choice questions: a medical student perspective

“…We read with considerable interest the study by Rahma et al that aimed to assess the impact of varying the number of distractors in multiple-choice questions (MCQs) 1. It was noticeable that in reducing the number of distractors from four to three, the discriminatory power of the examination increased, while the number of nonfunctional distractors was reduced.…”

“…Furthermore, students may rely on cues and implausible distractors,3 circumventing the need for thorough understanding. It should also be noted that Rahma et al and Sam et al both recognized that creating four good distractors may be challenging for examiners, especially when core knowledge is being tested 1,3. Therefore, VSAs may benefit both the examiner and examinee.…”

Objetos contemporâneos para ensino-aprendizagem da ressuscitação cardiopulmonar

“…Neste estudo optou-se por elaborar um questionário com 20 QME sobre RCP no adulto em SBV com o uso do DEA no ambiente hospitalar de acordo com as diretrizes da AHA (2015), utilizou-se os critérios do Conselho Nacional de Examinadores Médicos, publicado no manual "Construindo perguntas teste escrito para as ciências básicas e clínicas" (CASE; SWANSON, 2002). As questões foram estruturadas de acordo com o conteúdo exposto na videoaula e no vídeo de simulação, a fim de possibilitar o levantamento da aprendizagem de modo sistemático, abrangente e profundo.A QME de alta qualidade requer experiência e conhecimento sobre a adequada metodologia de elaboração(RAHMA et al, 2017, ABDULGHANI et al, 2015, pois o objetivo é o preparo de questões sem distratores para a obtenção de resultados que reflitam com precisão o nível de conhecimento(RUSH et al, 2016). É necessário que a QME seja validada por expertises, garantindo a qualidade necessária para a aplicação e tornando-se uma ferramenta adequada para a avaliação(BOTELHO et al, 2017;MAFINEJAD et al, 2017;ALCONERO-CAMARERO et al, 2016;VEGADA et al, 2016;RUSH et al, 2016;BUSTRAAN et al, 2016).As QME foram avaliadas em relação à organização, objetividade e clareza.…”

“…Avaliadas positivamente pelos expertises, sendo 96,89% em organização, 90,94% em objetividade e 88,75% em clareza. As sugestões e apontamentos realizados pelos expertises, foram avaliados e acatados, assim, considera-se que tornou-se uma ferramenta de qualidade e adequada para a avaliação sobre RCP no adulto em SBV com uso do DEA no ambiente hospitalar.A QME considerada adequadamente construída consiste em um enunciado/pergunta, podendo ser um caso clínico, seguido de quatro opções de escolha, uma correta e três incorretas (CASO;SWANSON, 2002), sendo este trabalho um desafio(ABDULGHANI et al, 2017) Rahma et al (2017). compararam o uso de três ou quatro opções em QME e identificaram que o uso de três alternativas é confiável, e sem diferença significativa na dificuldade, recomendam o uso de três ou quatro alternativas, com questões de alta qualidade Vegada et al (2016).…”