Formula Scoring of Multiple‐Choice Tests (Correction for Guessing)

Abstract

Formula scoring is a procedure designed to reduce multiple‐choice test score irregularities due to guessing. Typically, a formula score is obtained by subtracting a proportion of the number of wrong responses from the number correct. Examinees are instructed to omit items when their answers would be sheer guesses among all choices but otherwise to guess when unsure of an answer. Thus, formula scoring is not intended to discourage guessing when an examinee can rule out one or more of the options within a multiple‐choice item. Examinees who, contrary to the instructions, do guess blindly among all choices are not penalized by formula scoring on the average; depending on luck, they may obtain better or worse scores than if they had refrained from this guessing. In contrast, examinees with partial information who refrain from answering tend to obtain lower formula scores than if they had guessed among the remaining choices. (Examinees with misinformation may be exceptions.) Formula scoring is viewed as inappropriate for most classroom testing but may be desirable for speeded tests and for difficult tests with low passing scores. Formula scores do not approximate scores from comparable fill‐in‐the‐blank tests, nor can formula scoring preclude unrealistically high scores for examinees who are very lucky.

Optimal correction for guessing in multiple-choice tests

“…Frary, 1988) recommend not penalizing for wrong answers. They claim that psychological factors or personal characteristics may influence the decision of students to omit questions on which they have partial knowledge and a positive expected reward from answering.…”

“…1 The debate was surveyed by Diamond and Evans (1973), who examined the debate in terms of validity, reliability, test-taker behavior and other dimensions. Frary (1988) views formula scoring as "inappropriate for most classroom testing but desirable for speeded tests and for difficult test with low passing scores." Bar-Hillel, Budescu, and Attali (2005) suggest that using formula scoring is a form of irrationality on the part of the examiner, as they claim number right is a superior scoring method.…”

Efeito da penalização em itens dicotômicos no ensino de Física

“…Essa questão acerca dos acertos ao acaso, e a consequente elevação do escore final do respondente, mesmo quando este sabe pouco sobre o assunto,é uma preocupação bastante frequente entre pesquisadores [6,9,10]. O "chute" exerce influência sobre a fidedignidade das provas, que, por sua vez, está intimamente ligadaà atribuição de escores que representem acuradamente a proficiência do respondente no domínio avaliado [11][12][13] .…”

The cognitive plausibility of statistical classification models: Comparing textual and behavioral evidence

“…are they making an intentional informed choice or just guessing), a similar approach can be taken in linguistics when analysing behavioral data in order to increase the validity of the results. Klavan (2014) used a strategy referred to as formula scoring (Frary 1988) 27.7/60 = 46% * Weakest Baseline Chance = 410/60 ( ) = 160.6% * Best Baseline = 20/60 ( ) = 33.3% Table 3. Agreement between the corpus models and native speakers in the four experimental case studies…”