The Impact of Item Deletion on Equating Conversions and Reported Score Distributions

Abstract

A formal analysis of the effects of item deletion on equating/scaling functions and reported score distributions is presented. There are two components of the present analysis: analytical and empirical. The analytical decomposition demonstrates how the effects of item characteristics, test properties, individual examinee responses, and rounding rules combine to produce the item deletion effect on the equating/scaling function and candidate scores, In addition to demonstrating how the deleted item's psychometric characteristics can affect the equating function, the analytical component of the report examines the effects of not scoring versus scoring all options correct, the effects of re‐equating versus not re‐equating, and the interaction between the decision to re‐equate or to not re‐equate and the scoring option chosen for the flawed item. The empirical portion of the report uses data from the May 1982 administration of the SA T, which contained the circles item, to illustrate the effects of item deletion on reported score distributions and equating functions. The empirical data verify what the analytical decomposition predicts.

Item Response Theory

“…Lord and Wild (1985) compared the contribution of the four verbal item types to measurement accuracy of the GRE General Test, finding that the reading comprehension item type measures something slightly different from what is measured by sentence completion, analogy, or antonym item types. Dorans (1986) used IRT to study the effects of item deletion on equating functions and the score distribution on the SAT, concluding that reequating should be done when an item is dropped. Kingston and Holland (1986) compared equating errors using IRT and several other equating methods, and several equating designs, for equating the GRE General Test, with varying results depending on the specific design and method.…”

Item Response Theory

“…Lord and Cheryl Wild (1985) compared the contribution of the four verbal item types to measurement accuracy of the GRE General Test, finding that the reading comprehension item type measures something slightly different from what is measured by sentence completion, analogy, or antonym item types. Dorans (1986) used IRT to study the effects of item deletion on equating functions and the score distribution on the SAT, concluding that reequating should be done when an item is dropped. Kingston and Holland (1986) compared equating errors using IRT and several other equating methods, and several equating designs, for equating the GRE General Test, with varying results depending on the specific design and method.…”

NCME 2008 Presidential Address: The Impact of Anchor Test Configuration on Student Proficiency Rates

“…The comparison tells us what the relationship is between the score scale of the new test and the score scale used previously. We use this information to identify the corresponding scores on the old and new tests (Dorans, 1986;Haertel, 2004 Table 2 shows that the tests were long and assessed a variety of subject areas and grades. They had been rigorously developed and subjected to extensive internal and external reviews.…”

“…The comparison tells us what the relationship is between the score scale of the new test and the score scale used previously. We use this information to identify the corresponding scores on the old and new tests (Dorans, 1986; Haertel, 2004). Once this is done, we can compare old and new test scores, apply previously established cut scores to the new test, and the like.…”