The “None of the Above” Option in Multiple-Choice Testing: An Experimental Study

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

doi:10.1080/00220973.2013.795127

Cited by 24 publications

(29 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most studies demonstrate no effect of NOTA on discrimination, either as the key or as a distractor (Crehan & Haladyna, 1991;Dudycha & Carpenter, 1973;Tollefson, 1987). However, when students are asked to provide the correct response when choosing NOTA and their performance is rescored with regard to the accuracy of this response, including NOTA as a distractor does decrease discrimination coefficients (DiBattista, Sinnige-Egger, & Fortuna, 2014). Such a result suggests that previous work exploring the influence of NOTA on discrimination coefficients may have underestimated its true effect by pooling together NOTA-responders with and without accurate knowledge.…”

Section: Introductionmentioning

confidence: 99%

A Systematic Assessment of ‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom

2015

Self Cite

View full text Add to dashboard Cite

Multiple choice writing guidelines are decidedly split on the use of ‘none of the above’ (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect on difficulty and discrimination. In these studies, NOTA commonly yields increased difficulty when it is used as the correct response, and no change in discrimination regardless of usage. However, when NOTA is implemented as a distractor, rarely is consideration given to the distractor that could have been written in its place. Here, we systematically replaced each distractor in a series of questions with NOTA across different versions of an Introductory Psychology examination. This approach allowed us to quantify the quality of each distractor based on its relative discrimination index and assess the effect of NOTA relative to the quality of distractor it replaced. Moreover, our use of large Introductory Psychology examinations afforded highly stable difficulty and discrimination estimates. We found that NOTA increased question difficulty only when it was the correct response, with no effect on difficulty of replacing any distractor type with NOTA. Moreover, we found that NOTA decreased discrimination when it replaced the most effective distractors, with no effect on discrimination of replacing either the correct response or lowest quality distractor with NOTA. These results replicate the common finding that inclusion of NOTA as the correct response increases question difficulty by equally luring high-performing and low-performing students toward distractors. Moreover, we have shown that including NOTA as a distractor can reduce discrimination if used in lieu of a well written alternative, suggesting that multiple choice authors should avoid using NOTA on multiple choice tests. Les guides de rédaction pour les questions à choix multiple sont incontestablement partagés sur l’usage de la réponse « aucune des situations ci-dessus ». Certains auteurs déconseillent de l’employer alors que d’autres préconisent de le faire. De plus, des études empiriques de l’emploi de cette expression ont mené à des résultats mitigés. En général, ces études ont utilisé l’option « aucune des situations ci-dessus » soit comme étant la réponse correcte soit comme distracteur et ont évalué ses effets sur la difficulté et le discernement. Dans ces études, la réponse « aucune des situations ci-dessus » mène généralement à une augmentation de la difficulté quand l’option est employée comme étant la réponse correcte et il n’y a aucun changement en ce qui concerne le discernement, quel que soit son usage. Toutefois, quand ce type d’option de réponse est utilisé en tant que distracteur, on trouve rarement une justification à l’emploi d’un distracteur qui aurait pu être utilisé à sa place. Dans le cas présent, nous avons systématiquement remplacé chaque distracteur dans une série de questions contenant l’option « aucune des situations ci-dessus » dans différentes versions d’un examen d’introduction à la psychologie. Cette approche nous a permis de quantifier la qualité de chaque distracteur sur la base de l’indice de son discernement relatif et d’évaluer les effets de l’option relative sur la qualité du distracteur qu’elle remplaçait. De plus, puisque nous avons utilisé de grands examens d’introduction à la psychologie, cela nous a permis de faire des estimations de la difficulté et du discernement hautement stables. Nous avons trouvé que l’emploi de l’option « aucune des situations ci-dessus » augmentait la difficulté de la question seulement lorsque cette option était la réponse correcte et qu’il n’avait aucun effet sur la difficulté présente lorsqu’on remplaçait n’importe quel type de distracteur par l’option « aucune des situations ci-dessus ». En outre, nous avons trouvé que l’emploi de l’option « aucune des situations ci-dessus » diminuait le discernement quand elle remplaçait les distracteurs les plus efficaces et qu’elle n’avait aucun effet sur le discernement quand elle remplaçait soit la réponse correcte soit le distracteur le moins plausible par l’option « aucune des situations ci-dessus ». Ces résultats reproduisent les conclusions communes selon lesquelles l’emploi de l’option « aucune des situations ci-dessus » comme étant la réponse correcte augmente la difficulté de la question car dans ce cas, tant les étudiants brillants que les étudiants médiocres sont leurrés de façon identique vers les distracteurs. Par surcroît, nous avons montré que le fait d’inclure l’option « aucune des situations ci-dessus » en tant que distracteur pouvait réduire le discernement si on l’utilisait à la place d’une alternative de réponse bien rédigée, ce qui suggère que les auteurs de questions à choix multiples devraient éviter d’utiliser l’option « aucune des situations ci-dessus » dans les examens à choix multiples.

show abstract

Section: Introductionmentioning

confidence: 99%

A Systematic Assessment of ‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…So, the instructive investigators proposed that "none of the above" should be excluded from all health sciences MCQs tests. [30,31] Nurse educators have to recognize the effects of this flaw on students and on the assessment results. It can be hurt achiever students more than failed ones.…”

Section: Discussionmentioning

confidence: 99%

Development of multiple choice questions' instructional plan for the nursing educators

Ibrahim¹

2019

JNEP

View full text Add to dashboard Cite

Multiple-choice questions (MCQs) are commonly employed tests configuration in healthcare sciences for several years. They are efficient means for formative and summative evaluation of the students in nursing education. If the MCQs are designed competently, it will be a resourceful measurement for a valid assessment of the nursing students. The present study aimed to evaluate the construction quality of MCQs and its common item flaws in core nursing subjects at Faculty of Nursing, then develop an instructional plan for MCQs construction (one best answer format) to guide nursing educators. The study passed through exploratory-descriptive and methodological research designs at the Faculty of Nursing, Alexandria University, Egypt. Two samples were selected: First, 253 MCQs were selected from the different final exams (2017-2018) of the core twelve nursing subjects; Second, 21 academic nursing educators for evaluating of the suggested instructional plan. Both samples were chosen by convenience sampling technique. MCQs Assessment Form (MCQAF) was the first tool used to assess the construction quality of MCQs development and its items flaws. The second tool was MCQs' Instructional Plan Evaluation Sheet (MCQ IPES) which measured the content and face validity of the suggested instructional plan. The results revealed that 45.5% (115 MCQs) of the study sample contained ten item flaws. As regards the assessment of MCQs construction quality, evidently, the majority of nursing subject's exams has mean scores around a satisfactory level in MCQs quality construction. After submitting the developed plan to the expert group, almost the entire group found that the developed plan is accurate, with sound information, considered the useful and valued educational resource. Moreover, it has appropriate content in vocabulary, sentence structure, grammar, and concepts. Likewise, the experts reported that the developed plan is clear enough to be used by nursing educators and it is attractive and interesting self-reference tool. Conclusion & recommendation: less than one half of the study MCQs has ten item flaws and almost all of them obtained satisfactory mean scores in MCQs quality construction. The developed instructional plan is a beginning and beneficial step for nursing educators and considered an instructional mean for MCQs construction guidance. Further studies are application of the developed plan among nursing educators and investigation of the awareness and compliance of nursing educators with the MCQs construction rules is needed.

show abstract

“…Other research into MCQ test quality have identified the importance of the placement of options in logical order (Huntley and Welch, 1993), and the avoidance of "none of the above statements" (Crehan and Haladyna, 1991;DiBattista et al, 2014) although the effect of unfocused stems is limited (Downing et al, 1991).…”

Section: Introductionmentioning

confidence: 99%

Do Item Writing Best Practices Improve Multiple Choice Questions for University Students?

et al. 2018

View full text Add to dashboard Cite

We conduct a randomized experiment using a series of three finance exams sat by over 1,000 students at an Australian university, to determine the extent to which multiple-choice questions (MCQs) are improved by item writing best practices. We identify deviations from best practices (i.e. "flaws") in MCQs, correct these flaws in a random sample of MCQs and re-administer the exams. Regardless of question difficulty, editing the MCQs according to best practices provides greater clarity for students and increases the proportion of correct responses. We also observe a substantial improvement in MCQ performance related to the effective removal of non-performing distractors. The effect of MCQ editing is larger on those items that students find more difficult. The discriminatory power of MCQs edited according to best practices declines on average, but it increases among students with the lowest scores.

show abstract

The “None of the Above” Option in Multiple-Choice Testing: An Experimental Study

Cited by 24 publications

References 41 publications

A Systematic Assessment of ‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom

A Systematic Assessment of ‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom

Development of multiple choice questions' instructional plan for the nursing educators

Do Item Writing Best Practices Improve Multiple Choice Questions for University Students?

Contact Info

Product

Resources

About