Multiple-Choice Testing: Question and Response Position

Bresnock, Anne E.; Graves, Philip E.; White, Nancy

doi:10.1080/00220485.1989.10844626

Cited by 18 publications

(17 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bresnock et al (1989) reported that students had better outcomes when writing four-choice MCQ exams if a higher proportion of the correct answers were represented by the letter A rather than the letter D. They postulated that students were more prone to recognize the correct answer if they saw it immediately rather than after reading through several distractors and possibly becoming confused. On the other hand, a pair of studies separated in time by almost 50 years and each controlled to remove knowledge as a basis for answer selection revealed that a higher proportion (70-80%) of study participants, chose central letters (B and C) rather than A or D, the letters at the beginning or the end of the list (Attali & Bar-Hillel, 2003;Berg & Rapaport, 1954).…”

Section: Discussionmentioning

confidence: 99%

“…However, the latter effect was noticeable only at a level of p < 0.10, a level not usually used as a benchmark for significance. In contrast, a number of other studies comparing sequential and random ordering of questions and involving a variety of disciplines did not find it advantageous for students to answer exam questions in the same order that they had encountered the content during lectures, be they students of large-or small-enrolment classes (Bresnock et al, 1989;Kagundu & Ross, 2015;Khan et al, 2013;Neely, Springston, & McCann, 1994;Sue, 2009;Tal, Akers, & Hodge, 2008).…”

Section: Introductionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

“…In an effort to reduce the ability of students to copy from one another, several exam versions are often created via the process of MCQ scrambling (Bresnock, Graves, & White, 1989;Khan et al, 2013). This scrambling can focus on two possible examination parameters: the arrangement of the individual questions composing that exam can be shuffled and/or the order of the answer choices linked with each question can be rearranged (Bresnock et al, 1989;Khan et al, 2013;Sue, 2009). An important benefit of MCQ reorganization, be it by the order of the questions and/or by the answer choices, is that it makes the summative approach to student evaluation as fair as possible by allowing all students to be assessed using the same questions (Khan et al, 2013;Sue, 2009).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Carnegie

2017

cjsotl-rcacea

View full text Add to dashboard Cite

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when preparing multiple exam versions is that they must be equivalent in their assessment of student knowledge. This project investigated a possible influence of correct answer organization on student answer selection when writing multiple versions of MCQ exams. The specific question asked was whether the existence of a series of four to five consecutive MCQs in which the same letter represented the correct answer had a detrimental influence on a student’s ability to continue to select the correct answer as he/she moved through that series. Student outcomes from such exams were compared with results from exams with identical questions but which did not contain such series. These findings were supplemented by student survey data in which students self-assessed the extent to which they paid attention to the distribution of correct answer choices when writing summative exams, both during their initial answer selection and when transferring their answer letters to the Scantron sheet for correction. Despite the fact that more than half of survey respondents indicated that they do make note of answer patterning during exams and that a series of four to five questions with the same letter for the correct answer would encourage many of them to take a second look at their answer choice, the results pertaining to student outcomes suggest that MCQ randomization, even when it does result in short serial arrays of letter-specific correct answers, does not constitute a distraction capable of adversely influencing student performance. Dans les très grandes classes de cours de première et deuxième années, l’évaluation sommative se déroule souvent par le biais d’examens comportant des questions à choix multiples afin de pouvoir donner rapidement les résultats aux étudiants. Plusieurs versions de ces examens sont souvent préparées et les questions sont brouillées par ordinateur pour dissuader la tricherie. Lors de la préparation de plusieurs versions d’un examen à choix multiples, l’un des paramètres importants à prendre en considération est que chaque version doit être semblable aux autres pour évaluer équitablement les connaissances des étudiants. Ce projet a pour but d’examiner l’influence possible de l’organisation des réponses correctes sur le choix des réponses des étudiants lors de la préparation de plusieurs versions d’un examen à choix multiples. La question spécifique qui a été posée était de savoir si l’existence d’une série de quatre ou cinq questions à choix multiples consécutives pour lesquelles la même lettre représentait la bonne réponse pouvait avoir une influence préjudiciable sur l’aptitude des étudiants à continuer à choisir la bonne réponse alors qu’ils progressent d’une question à l’autre dans la même série. Les résultats des étudiants qui passent de tels examens ont été comparés aux résultats obtenus quand les étudiants passent des examens dont les questions sont les mêmes mais qui ne comportent pas de telles séries. Ces résultats ont été enrichis par les réponses à une enquête auprès des étudiants pour laquelle les étudiants ont été auto-évalués concernant la question de savoir s’ils avaient remarqué la répartition des réponses correctes parmi les choix multiples quand ils passaient des examens sommatifs, à la fois au départ, quand ils choisissaient leurs réponses, et ensuite quand ils transféraient les lettres correspondant à leurs réponses sur la feuille Scanton pour la correction. Malgré le fait que plus de la moitié des répondants aient indiqué qu’ils ne font pas attention à la structuration des réponses pendant l’examen et qu’une série de quatre ou cinq questions ayant la même lettre pour la bonne réponse pourrait encourager beaucoup d’entre eux à regarder de plus près leur choix de réponse, la conclusion concernant les résultats obtenus par les étudiants suggère que la randomisation des questions à choix multiples, même quand elle aboutit à des séries de réponses correctes identifiées par la même lettre, ne constitue pas une distraction capable d’influencer négativement le rendement des étudiants.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Carnegie

2017

cjsotl-rcacea

View full text Add to dashboard Cite

show abstract

“…19 This observation may correspond to intuition about test taking. Students who see the correct answer first may take less time looking at the other possible answers on those items.…”

Section: ■ Literature Reviewmentioning

confidence: 99%

Investigating Factors That Influence Item Performance on ACS Exams

2012

View full text Add to dashboard Cite

General chemistry tests from the Examinations Institute of the Division of Chemical Education of the American Chemical Society have been analyzed to identify factors that may influence how individual test items perform. In this paper, issues of item order (position within a set of items that comprise a test) and answer order (position of correct answer relative to incorrect distractors) are discussed. Answer order is identified as potentially important, particularly for conceptually based items. When the correct answer appears earlier among the answer choices, there is some greater propensity for student performance to be better. Item-order effects are also possible, particularly when students encounter several challenging items consecutively. Performance on the next item may be lower than expected, possibly because of cognitive-load effects. In this paper, issues of item order (position within a set of items that comprise a test) and answer order (position of correct answer relative to incorrect distractors) are discussed. Answer order is identified as potentially important, particularly for conceptually based items. When the correct answer appears earlier among the answer choices, there is some greater propensity for student performance to be better. Item-order effects are also possible, particularly when students encounter several challenging items consecutively. Performance on the next item may be lower than expected, possibly because of cognitive-load effects.

show abstract

Correlates of student performance in Business and Economics Statistics

Krieg

Uyar

1997

J Econ Finance

View full text Add to dashboard Cite

Multiple-Choice Testing: Question and Response Position

Cited by 18 publications

References 18 publications

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Investigating Factors That Influence Item Performance on ACS Exams

Correlates of student performance in Business and Economics Statistics

Contact Info

Product

Resources

About