Screening for Potentially Biased Items in Testing Programs

Abstract

What statistical methods are being used to identify biased test items? What are the advantages and disadvantages of each method? What sample sizes are required to use each method? Which methods are most useful to the practitioner?

Assessing Differential Item Functioning in Performance Assessment: Review and Recommendations

“…As a consequence, by the end of the 1980s a large gap separated the technical developments in DIF research and the applied needs of the testing community. To meet this demand, Hills (1989) published a timely, concise account of DIF detection techniques for dichotomously scored items, focusing on the comparative utility of the various methods. Following Hills' article, Holland and Wainer (1993), and Camilli and Shepard (1994) provided an elaborate summary of the theory and application of modern bias detection techniques.…”

Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the Mantel–Haenszel and logistic regression techniques

“…Studies such are those by Hills (1990), Mazor et al (1992), Zieky (1993), Rogers and Swaminathan (1993), Muñiz et al (2001) and Fidalgo et al (2004) have examined at small group situations, while those by Roussos and Stout (1996), Miller and Oshima (1992) and Narayanan and Swaminathan (1996) have compared the effect of different group sizes on several detection procedures. The findings of the recent review by Herrera et al (2005) favour the use of MH with small groups.…”

A Generalized DIF Effect Variance Estimator for Measuring Unsigned Differential Test Functioning in Mixed Format Tests

“…Formally defined, DIF exists in an item if “… test takers of equal proficiency on the construct intended to be measured by a test, but from separate subgroups of the population, differ in their expected score on the item” (Roussos & Stout, 2004, p. 107). A variety of DIF detection procedures have been developed for dichotomous (Camilli & Shepard, 1994; Clauser & Mazor, 1998; Hills, 1989) and polytomous items (Penfield & Lam, 2000; Potenza & Dorans, 1995). While DIF concerns invariance of measurement properties at the level of the individual item, at times it is of interest to examine the aggregated effect of DIF across the items of either the entire test or a subset of test items.…”