The Effects of Referent Item Parameters on Differential Item Functioning Detection Using the Free Baseline Likelihood Ratio Test

Abstract

The purpose of this simulation study is to investigate the effects of anchor subtest composition on the accuracy of item response theory (IRT) likelihood ratio (LR) differential item functioning (DIF) detection (Thissen, Steinberg, & Wainer, 1988). Here, the IRT LR test was implemented with a free baseline approach wherein a baseline model was formed by freeing all items except a referent or anchor subset and examining the changes in fit with respect to a series of models wherein 1 item at a time was constrained in addition to the referent(s). The results clearly indicated that the composition of the anchor subtest is important for accurate DIF detection. It was found that using a single highly discriminating rather than a low discriminating referent greatly enhanced the power of the procedure. Moreover, in conditions involving small DIF or smaller sample sizes or both, power appeared to improve when a group of highly discriminating referents was used. These findings have implications for applied research involving short scales and small sample sizes.

DIF Detection Using Multiple‐Group Categorical CFA With Minimum Free Baseline Approach

“…On the other hand, the free baseline approach in this study produces reasonable empirical Type I error rates. Similar results can be found in other DIF contamination studies (Finch, ; Lopez Rivas et al., ; Shealy & Stout, ; Stark et al., ; Wang & Yeh, ; Woods, ). Moreover, MCCFA models with the constrained baseline approach cannot effectively handle the situation of more than one DIF items due to DIF contamination.…”

“…However, “normal” unique factor variances are commonly used in MCCFA, and by doing so, the multiple‐group GRM is only an approximate model for MCCFA with a proper rescaling. Second, in multiple‐group GRM, one usually assumes all the parameters for anchor item(s) to be the same between groups for linking and identification purposes (Lopez Rivas et al., ). In contrast, our current study illustrates the effect in efficiency, validity, and advantage of the DIF test while relaxing the number of constraints to the minimum, that is, with only partial parameters for one single item set equal for identification.…”

“…In the more restricted model, parameters of the tested item are assumed to be the same across groups, whereas they are allowed to differ in the other model. Depending on how all but the tested items are treated, such a nested‐model approach can be further divided into two categories, namely, the constrained baseline and free baseline approaches (Lopez Rivas et al., ; Stark et al., ).…”

“…Note that for the free baseline approach, the usual strategy for the common metric is to select one or some referent item(s) as anchor, and their parameters are assumed the same across groups (Lopez Rivas et al., ; Stark et al., ; Wang & Yeh, ). However, even with the one‐item anchor scenario, setting all parameters of the anchor item to be the same is actually more than necessary for model identifiability in MCCFA.…”

“…Instead of the constrained baseline approach, Lopez Rivas, Stark, and Chernyshenko (), Stark et al. (), Wang (), and Wang and Yeh () recommend the free baseline approach (or constant anchor method) that allows item parameters to be “freely” estimated except constraining a subset of items across groups as referent.…”