Measurement Invariance and Differential Item Functioning Across Gender Within a Latent Class Analysis Framework: Evidence From a High-Stakes Test for University Admission in Saudi Arabia

Abstract

The main aim of the present study was to investigate the presence of Differential Item Functioning (DIF) using a latent class (LC) analysis approach. Particularly, we examined potential sources of DIF in relation to gender. Data came from 6,265 Saudi Arabia students, who completed a high-stakes standardized admission test for university entrance. The results from a Latent Class Analysis (LCA) revealed a three-class solution (i.e., high, average, and low scorers). Then, to better understand the nature of the emerging classes and the characteristics of the people who comprise them, we applied a new stepwise approach, using the Multiple Indicator Multiple Causes (MIMIC) model. The model identified both uniform and non-uniform DIF effects for several items across all scales of the test, although, for the majority of them, the DIF effect sizes were negligible. Findings from this study have important implications for both measurement quality and interpretation of the results. Particularly, results showed that gender is a potential source of DIF for latent class indicators; thus, it is important to include those direct effects in the latent class regression model, to obtain unbiased estimates not only for the measurement parameters but also of the structural parameters. Ignoring these effects might lead to misspecification of the latent classes in terms of both the size and the characteristics of each class, which in turn, could lead to misinterpretations of the obtained latent class results. Implications of the results for practice are discussed.

Gender-Based Differential Item Function for the Positive and Negative Semantic Dimensions of the Relationship Satisfaction Scale with Item Response Theory

“…Still, an investigation of the invariance for each item is warranted because the level of information contained in the individual items in the PN-SMD was found to be inadequate [13], and the model fits reported in previous studies were also unsatisfactory [23], though the internal consistency was good (α = 0.92-0.93) [24]. When employing a MIMIC model, both the measurement model and the structural model can be used to evaluate the direct effect of a covariate that defines group membership (e.g., gender) on factor means and factor indicators (items) [31]. This method can help to identify any gender-based DIF traits in the PN-SMD.…”

“…A more recent and popular approach is multiple indicators and multiple causes (MIMIC) modeling for the detection of DIF [30]. The MIMIC model is a specialized version of structural equation modeling (SEM) that incorporates causal variables, or covariates, into a confirmatory factor analysis model [31]. According to Cheng, Shao and Lathrop [30], DIF can be understood as a model mediated by groups.…”

Analyzing Measurement Invariance for Studying the Gender Gap in Educational Testing

“…These findings are consistent with some advantages that studies reported for some of these methods. For example, Tsaousis et al (2020) mentioned that LCA could "attempt to identify possible sources of DIF across the covariate's levels" (p. 3). On the other hand, some methods were not used by any of the studies from Category C. These are area measures, IRT difficulty parameters comparison, and logits of gender-specific item-difficulty scores.…”

Differential item functioning across gender with MIMIC modeling: PISA 2018 financial literacy items

“…There are studies stating that MIMIC models are more useful compared to other techniques such as multigroup CFA in examining DIF (Vandenberg & Lance, 2000;Millsap, 2011). MIMIC modeling contributes to external validity by examining the relationship between covariate and latent structure, and to internal validity by estimating IRT parameters (Tsaousis et al, 2020). MIMIC modeling allows us to see the effect of covariates.…”

“…Hence, the aim of this study is to investigate the presence of DIF over the gender variable with a stepwise procedure conducted with a MIMIC modeling framework that has been developed by Masyn, (2017). The MIMIC approach is a method to test measurement invariance, and since its introduction (Masyn, 2017), a study conducted with real data by Tsaousis et al, (2020) but there is no study with international large scale assessment data in which this method was used. Consequently, in this study, following the stepwise procedure outlined by Masyn (2017), to explore sources of DIF over gender using large scale assessment data (i.e., PISA 2018 financial literacy test).…”