Linking Methods for the Zinnes–Griggs Pairwise Preference IRT Model

Abstract

Forced-choice item response theory (IRT) models are being more widely used as a way of reducing response biases in noncognitive research and operational testing contexts. As applications have increased, there has been a growing need for methods to link parameters estimated in different examinee groups as a prelude to measurement equivalence testing. This study compared four linking methods for the Zinnes and Griggs (ZG) pairwise preference ideal point model. A Monte Carlo simulation compared test characteristic curve (TCC) linking, item characteristic curve (ICC) linking, mean/mean (M/M) linking, and mean/sigma (M/S) linking. The results indicated that ICC linking and the simpler M/M and M/S methods performed better than TCC linking, and there were no substantial differences among the top three approaches. In addition, in the absence of possible contamination of the common (anchor) item subset due to differential item functioning, five items should be adequate for estimating the metric transformation coefficients. Our article presents the necessary equations for ZG linking and provides recommendations for practitioners who may be interested in developing and using pairwise preference measures for research and selection purposes.

Linking Methods for Multidimensional Forced Choice Tests Using the Multi-Unidimensional Pairwise Preference Model

“…For the linking coefficient A, the ICC and M/M methods showed comparable results, whereas for K, the ICC method outperformed the M/M method across the simulation conditions. The superior performance of the ICC method accords with results reported in previous studies with unidimensional ideal point models (Lee et al, 2017;Seybert et al, 2014).…”

An Extension of Testlet-Based Equating to the Polytomous Testlet Response Theory Model

“…The ability population distribution for the base form is the same as the b -parameter population distribution to assure that the difficulty of test is appropriate for the examinees ( Tao and Cao, 2016 ). The ability population distribution for the new form with a slightly higher mean than the base form to reflect ability differences between two groups ( Lee et al, 2016 ; Andersson, 2018 ). For the three pairs of test forms, the testlet effect indexed by the for the base form and the reference form were drawn from three uniform distributions: (0.1, 0.5), (0.6, 1), and (1.1, 2.0) corresponding to low, moderate and high levels of LID, respectively ( Wang et al, 2002 ; DeMars, 2006 ; Cao et al, 2014 ; Tao and Cao, 2016 ).…”

Two IRT Characteristic Curve Linking Methods Weighted by Information

“…\end{array} \end{equation}$$where $${w_i}$$ refers to the weight assigned to person i. For convenience, we adopt 161 evenly distributed ability points (see also Lee et al., 2017) from −4.00 to 4.00 with an increment of 0.05 and set $${w_i} = 1$$ in this article. This means that all persons are treated equally when calculating the loss function, which is in line with the software settings in equateIRT (Battauz, 2015), plink (Weeks, 2010), and sirt (Robitzsch, 2020).…”

“…}}{{\rm{1}}^2}} )$, and $$N( {0.5,{\rm{ 1}}{\rm{. }}{{\rm{2}}^2}} )$$, in order to construct linking conditions with no, small, and moderate ability differences, respectively (Andersson, 2018; Kim, 2015; Lee et al., 2017). Thus, the true linking constants were $$A = 1$$, $$B = 0$$; $$A = 1.1$$, $$B = 0.2$$; and $$A = 1.2$$, $$B = 0.5$$, respectively, for the three new‐form populations.…”

“…The ability difference between linking groups, sample size per group, and test length was manipulated. Persons taking the old form were sampled from N (0, 1) and persons taking the new form were sampled from three distributions, which were N (0, 1), N (0.2, 1.1 2 ), and N (0.5, 1.2 2 ), in order to construct linking conditions with no, small, and moderate ability differences, respectively (Andersson, 2018;Kim, 2015;Lee et al, 2017). Thus, the true linking constants were A = 1, B = 0; A = 1.1, B = 0.2; and A = 1.2, B = 0.5, respectively, for the three new-form populations.…”