An R Toolbox for Score-Based Measurement Invariance Tests in IRT Models

Abstract

The detection of differential item functioning (DIF) is a central topic in psychometrics and educational measurement. In the past few years, a new family of score-based tests of measurement invariance has been proposed that allows the detection of DIF along arbitrary person covariates in a variety of item response theory (IRT) models. This paper illustrates the application of these tests within the R system for statistical computing, making them accessible to a broad range of users. This presentation also includes IRT models for which these tests have not previously been investigated, such as the generalized partial credit model. The paper has three goals: First, we review the ideas behind score-based tests of measurement invariance. Second, we describe the implementation of these tests within the R system for statistical computing, which is based on the interaction of the R packages mirt, psychotools and strucchange. Third, we illustrate the application of this software and the interpretation of its output in two empirical datasets, and show how to conduct simulation studies, such as IRT-based power analyses, in the context of DIF investigations. The complete R code for reproducing our results is reported in the paper and its appendix.

An Evaluation of DIF Tests in Multistage Tests for Continuous Covariates

“…If we sum these individual score contributions over groups of test takers, the corresponding cumulative sums should also fluctuate around zero; however, if the model is not accurate, the cumulative sums can deviate strongly from zero. For a more detailed but slightly more technical introduction, see [48].…”

“…This strategy can be applied with score-based tests. We do not provide details for conciseness, but details on how score-based tests can be applied to item parameter estimates can be found in the literature, e.g., [48] or [50].…”

Evaluating the quality of life impact of recurrent urinary tract infection: Validation and refinement of the Recurrent UTI Impact Questionnaire (RUTIIQ)

“…19,37 Finally, it is assumed that an item should be interpreted in the same way across different subgroups, known as item invariance. 38 Likelihood ratio χ 2 analysis was conducted to check for item invariance, or the absence of differential item functioning (DIF). 38 This examined the extent to which each item performs differently within the model based on biological sex (female vs. male), age (older vs. younger than the median, 42 years old), household income (> £25 000 vs. < £25 000), level of education (university degree or above vs. school or lower), and current antibiotic use (yes vs. no).…”

“…39 A statistically significant group difference (χ 2 , p adj < 0.05) indicated the presence of DIF. 38…”

Semi-automated Rasch analysis with differential item functioning

“…Erroneous ignorance of such biases leads to a biased instrument (Borsboom, 2006;. DIF assessment has become one of the standard ingredients of Rasch analysis and has been implemented in various ways, e.g., (Holland & Thayer, 1986;Swaminathan & Rogers, 1990;Kreiner & Christensen, 2011;Magis & Facon, 2013;Tutz & Schauberger, 2015;Komboz, Strobl, & Zeileis, 2018;Schauberger & Mair, 2020;Schneider, Strobl, Zeileis, & Debelak, 2021).…”