Overcorrection for Guessing in Multiple-Choice Test Scoring

Little, Ellis Beecher

doi:10.1080/00220671.1962.10882801

Cited by 12 publications

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The effects of the correction-for-guessing formula upon test characteristics have been studied; e.g., Glass and Wiley (1964) have demonstrated that the correction formula tends to decrease reliability, while Lord (1963) has shown that it tends to increase validity. It has been noted, though, by a variety of writers, that the assumptions underlying the correctior~ are generally invalid (e.g., Cureton, 1966;Shimuzu, 1965), and that the "correction" may be an overcorrection (e.g., West, 1923;Melbo, 1933;Little, 1962).…”

mentioning

confidence: 99%

THE PENALTY FOR NOT GUESSING¹

Slakter

1968

J Educational Measurement

View full text Add to dashboard Cite

mentioning

confidence: 99%

THE PENALTY FOR NOT GUESSING¹

Slakter

1968

J Educational Measurement

View full text Add to dashboard Cite

“…The formula is S = R – W/[2 (n – 1)]. This scoring method was first described by Little [ 182 ] in 1962.…”

Section: Resultsmentioning

confidence: 99%

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Kanzow¹,

Schmidt²,

Kanzow³

2023

JMIR Med Educ

View full text Add to dashboard Cite

Background Single-choice items (eg, best-answer items, alternate-choice items, single true-false items) are 1 type of multiple-choice items and have been used in examinations for over 100 years. At the end of every examination, the examinees’ responses have to be analyzed and scored to derive information about examinees’ true knowledge. Objective The aim of this paper is to compile scoring methods for individual single-choice items described in the literature. Furthermore, the metric expected chance score and the relation between examinees’ true knowledge and expected scoring results (averaged percentage score) are analyzed. Besides, implications for potential pass marks to be used in examinations to test examinees for a predefined level of true knowledge are derived. Methods Scoring methods for individual single-choice items were extracted from various databases (ERIC, PsycInfo, Embase via Ovid, MEDLINE via PubMed) in September 2020. Eligible sources reported on scoring methods for individual single-choice items in written examinations including but not limited to medical education. Separately for items with n=2 answer options (eg, alternate-choice items, single true-false items) and best-answer items with n=5 answer options (eg, Type A items) and for each identified scoring method, the metric expected chance score and the expected scoring results as a function of examinees’ true knowledge using fictitious examinations with 100 single-choice items were calculated. Results A total of 21 different scoring methods were identified from the 258 included sources, with varying consideration of correctly marked, omitted, and incorrectly marked items. Resulting credit varied between –3 and +1 credit points per item. For items with n=2 answer options, expected chance scores from random guessing ranged between –1 and +0.75 credit points. For items with n=5 answer options, expected chance scores ranged between –2.2 and +0.84 credit points. All scoring methods showed a linear relation between examinees’ true knowledge and the expected scoring results. Depending on the scoring method used, examination results differed considerably: Expected scoring results from examinees with 50% true knowledge ranged between 0.0% (95% CI 0% to 0%) and 87.5% (95% CI 81.0% to 94.0%) for items with n=2 and between –60.0% (95% CI –60% to –60%) and 92.0% (95% CI 86.7% to 97.3%) for items with n=5. Conclusions In examinations with single-choice items, the scoring result is not always equivalent to examinees’ true knowledge. When interpreting examination scores and setting pass marks, the number of answer options per item must usually be taken into account in addition to the scoring method used.

show abstract

“…In developing different test formats, researchers have given much thought to the question of how to correct scores for guessing to minimize its influence on test results (Albanese, 1986; Angoff & Schrader, 1984; Diamond & Evans, 1973; Frary, 1969, 1988; Hambrick, Meinz, & Oswald, 2007; Kolstad, Wagner, Kolstad, & Miller, 1983; Little, 1962; Lord, 1975). Although several techniques have been proposed and used; none has been found universally acceptable.…”

Section: Multiple-choice Testingmentioning

confidence: 99%

Knowledge assessment: Squeezing information from multiple-choice testing.

Nickerson

Butler

Carlin

2015

Journal of Experimental Psychology: Applied

View full text Add to dashboard Cite

Knowledge assessment via testing can be viewed from two vantage points: that of the test administrator and that of the test taker. From the administrator's perspective, the objective is to discover what an individual knows about a domain of interest. From that of the test taker, the challenge is to reveal what one knows. In this article we describe a procedure for administering and scoring multiple-choice tests that satisfies both of these objectives and we present experimental data that demonstrate its effectiveness. The method allows test takers to provide specific information about their confidence that each alternative for an item is the correct answer and makes guessing not only unnecessary but detrimental. From this information the administrator can derive measures of both knowledge and confidence, which, we argue, provides better estimates than systems that do not allow measurement of partial knowledge. The use of such measures for purposes of evaluation both of individual test takers' knowledge of a subject of interest and of the effectiveness of instruction with respect to that subject is discussed.

show abstract

Overcorrection for Guessing in Multiple-Choice Test Scoring

Cited by 12 publications

References 3 publications

THE PENALTY FOR NOT GUESSING¹

THE PENALTY FOR NOT GUESSING¹

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Knowledge assessment: Squeezing information from multiple-choice testing.

Contact Info

Product

Resources

About

Overcorrection for Guessing in Multiple-Choice Test Scoring

Cited by 12 publications

References 3 publications

THE PENALTY FOR NOT GUESSING1

THE PENALTY FOR NOT GUESSING1

Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods

Knowledge assessment: Squeezing information from multiple-choice testing.

Contact Info

Product

Resources

About

THE PENALTY FOR NOT GUESSING¹

THE PENALTY FOR NOT GUESSING¹