The Effect of Constructing Multiple-Choice Distractor Items around a Single Target Alternative

Abstract

The present research sought to determine whether the construction of multiple-choice alternatives based around a critical target answer would facilitate the selection of the target answer. Subjects were given a multiple-choice test consisting of 60 questions, each having four alternatives. Twenty of the 60 questions were the critical questions and were constructed to have no correct answer (i.e., asked nonsense) but appeared legitimate. One of the alternatives for the critical questions was the critical \ alternative, around which the other three distractor alternatives were derived. This was accomplished by systematically substituting each of the critical alternative's three components with another plausible component. This procedure produced a set of alternatives where the critical alternative was more similar to the other alternatives than they were to each other (i.e., it was the most prototypic). The results of two experiments using ranking and proportion scores showed a response bias effect: subjects selected the critical alternatives more often than would be expected by chance. Further analyses revealed that in lower ability subjects the effect disappeared when the critical alternatives were embedded in sets of distractors which had randomly ordered components. High ability subjects selected the critical alternatives more often than chance regardless of the distractors' component arrangement. The results suggest that test-makers should avoid constructing distractor alternatives around a correct alternative because the information provided in the set of alternatives may influence test-takers to select the target answer without any knowledge of the information being assessed.

Comprehension of Pictorial Symbols: Effects of Context and Test Method

“…Although the lack of plausibility can be obvious (as in the previous example), distractor quality in actual multiple-choice tests can be subtle and difficult to spot and may influence test performance (Katz & Lautenschlager, 1994;Wogalter & Marwitz, 1987;Wogalter, Marwitz, & Leonard, 1992).…”