Item Order Affects Performance on Multiple-Choice Exams

Balch, William R.

doi:10.1207/s15328023top1602_9

Cited by 33 publications

(29 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sequential order delivers the questions in the order that the content was covered during lectures whereas reverse sequencing does exactly the opposite. In a chapter contiguity exam, the questions are grouped by chapter, but within each chapter-related exam section, the questions are not in chronological order, and the chapter-batches themselves do not have to be inserted consecutively into the exam (Balch, 1989). Questions can also be ordered by increasing or decreasing level of difficulty (Noland, Russell & Madden, 2014;Perlini, Lind & Zumbo, 1998).…”

Section: Introductionmentioning

confidence: 99%

“…Balch (1989) compared all three approaches to exam construction within a large class of psychology students with the added constraint for the randomly-ordered exam that no two questions from the same chapter could immediately follow one another. His hypothesis was that factual information would be more easily retrieved if accessed in the order that it was first learned and encoded in long-term memory.…”

Section: Introductionmentioning

confidence: 99%

“…Indeed, higher-order learning involves the restructuring and organizing of new knowledge so that it can form a part of long-term memory by linking with that which is already known, in this way facilitating subsequent retrieval for application (Kirschner, 2002;Kirschner, Sweller, & Clark, 2006). Balch (1989) reported a small but significant difference in student outcomes between the sequential and chapter contiguity groups and suggested a slight beneficial effect for sequential compared to random. However, the latter effect was noticeable only at a level of p < 0.10, a level not usually used as a benchmark for significance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Carnegie

2017

cjsotl-rcacea

View full text Add to dashboard Cite

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when preparing multiple exam versions is that they must be equivalent in their assessment of student knowledge. This project investigated a possible influence of correct answer organization on student answer selection when writing multiple versions of MCQ exams. The specific question asked was whether the existence of a series of four to five consecutive MCQs in which the same letter represented the correct answer had a detrimental influence on a student’s ability to continue to select the correct answer as he/she moved through that series. Student outcomes from such exams were compared with results from exams with identical questions but which did not contain such series. These findings were supplemented by student survey data in which students self-assessed the extent to which they paid attention to the distribution of correct answer choices when writing summative exams, both during their initial answer selection and when transferring their answer letters to the Scantron sheet for correction. Despite the fact that more than half of survey respondents indicated that they do make note of answer patterning during exams and that a series of four to five questions with the same letter for the correct answer would encourage many of them to take a second look at their answer choice, the results pertaining to student outcomes suggest that MCQ randomization, even when it does result in short serial arrays of letter-specific correct answers, does not constitute a distraction capable of adversely influencing student performance. Dans les très grandes classes de cours de première et deuxième années, l’évaluation sommative se déroule souvent par le biais d’examens comportant des questions à choix multiples afin de pouvoir donner rapidement les résultats aux étudiants. Plusieurs versions de ces examens sont souvent préparées et les questions sont brouillées par ordinateur pour dissuader la tricherie. Lors de la préparation de plusieurs versions d’un examen à choix multiples, l’un des paramètres importants à prendre en considération est que chaque version doit être semblable aux autres pour évaluer équitablement les connaissances des étudiants. Ce projet a pour but d’examiner l’influence possible de l’organisation des réponses correctes sur le choix des réponses des étudiants lors de la préparation de plusieurs versions d’un examen à choix multiples. La question spécifique qui a été posée était de savoir si l’existence d’une série de quatre ou cinq questions à choix multiples consécutives pour lesquelles la même lettre représentait la bonne réponse pouvait avoir une influence préjudiciable sur l’aptitude des étudiants à continuer à choisir la bonne réponse alors qu’ils progressent d’une question à l’autre dans la même série. Les résultats des étudiants qui passent de tels examens ont été comparés aux résultats obtenus quand les étudiants passent des examens dont les questions sont les mêmes mais qui ne comportent pas de telles séries. Ces résultats ont été enrichis par les réponses à une enquête auprès des étudiants pour laquelle les étudiants ont été auto-évalués concernant la question de savoir s’ils avaient remarqué la répartition des réponses correctes parmi les choix multiples quand ils passaient des examens sommatifs, à la fois au départ, quand ils choisissaient leurs réponses, et ensuite quand ils transféraient les lettres correspondant à leurs réponses sur la feuille Scanton pour la correction. Malgré le fait que plus de la moitié des répondants aient indiqué qu’ils ne font pas attention à la structuration des réponses pendant l’examen et qu’une série de quatre ou cinq questions ayant la même lettre pour la bonne réponse pourrait encourager beaucoup d’entre eux à regarder de plus près leur choix de réponse, la conclusion concernant les résultats obtenus par les étudiants suggère que la randomisation des questions à choix multiples, même quand elle aboutit à des séries de réponses correctes identifiées par la même lettre, ne constitue pas une distraction capable d’influencer négativement le rendement des étudiants.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Carnegie

2017

cjsotl-rcacea

View full text Add to dashboard Cite

show abstract

“…Therefore, such examinations can cause biased measurement results both in favor and disfavor of examinees. As the item orders are different due to variance in forms, this alone could cause students to consider items "harder" or "easier" (Balch, 1989;Impara & Foster, 2006;Laffitte, 1984;Pettijohn & Sacco, 2007). Some reviews of test item order studies have concluded that item order does not influence student test performance (Barcikovski & Olsen, 1975;Carlson & Ostrosky, 1992;Gerov, 1980;Klosner & Gellman, 1973;Tippest & Benson, 1989).…”

mentioning

confidence: 99%

“…However, when research on item order in test forms is reviewed, it is clear that most studies have been based on Classical Test Theory (CTT). Some studies concluded that test item order influences test scores, item parameters, and completion time (Balch, 1989;Picou & Milhomme, 1997).…”

mentioning

confidence: 99%

Examining Differential Item Functions of Different Item Ordered Test Forms According to Item Difficulty Levels

2016

EDUC SCI-THEOR PRACT

View full text Add to dashboard Cite

The study aims to examine whether differential item function is displayed in three different test forms that have item orders of random and sequential versions (easy-to-hard and hard-to-easy), based on Classical Test Theory (CTT) and Item Response Theory (IRT) methods and bearing item difficulty levels in mind. In the correlational research, the data from a total of 578 seventh graders were gathered using an Atomic Structures Achievement Test. R programming language and "difR" package were employed for all the analyses. As a result of the analyses, it was concluded that a comparison of IRT-and CTT-based methods indicate a greater number of items with distinctively significant differential item functioning. Different item ordering leads students at the same ability levels to display different performances on the same items. As a result, it is found that item order differentiates the probability of correct response to the items for those at the same ability levels. A test form of sequential easy-to-hard questions brings more advantages than that of a hard-to-easy sequence or a random version. The findings show that it is essential to arrange tests that are employed to make decisions about people in consideration with psychometric principles.

show abstract

Shift Error Detection in Standardized Exams

Skiena

Sumazin

2000

Combinatorial Pattern Matching

View full text Add to dashboard Cite

Item Order Affects Performance on Multiple-Choice Exams

Cited by 33 publications

References 2 publications

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

Examining Differential Item Functions of Different Item Ordered Test Forms According to Item Difficulty Levels

Shift Error Detection in Standardized Exams

Contact Info

Product

Resources

About