Multiple-Group Noncompensatory Differential Item Functioning in Raju's Differential Functioning of Items and Tests

Oshima, T. C.; Wright, Keith; White, Nick

doi:10.1080/15305058.2015.1009980

Cited by 6 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To create the base group (BG), a random sample was drawn from the total sample of multiple groups and divided by the total number of groups being used in the multiple-group DIF (Oshima et al, 2015). By drawing a random sample of the average sample size, the base group becomes a representative sample (Oshima et al, 2015). In this method, all groups are treated equally and item parameter estimates are based on the base group's scale (Oshima et al, 2015).…”

Section: Measures and Variablesmentioning

confidence: 99%

“…By drawing a random sample of the average sample size, the base group becomes a representative sample (Oshima et al, 2015). In this method, all groups are treated equally and item parameter estimates are based on the base group's scale (Oshima et al, 2015). In our study, the total sample of N = 5712 was divided by five (i.e., the number of ethnicity groups) to yield a BG of n = 1142 students (5712/5 = 1142.2).…”

Section: Measures and Variablesmentioning

confidence: 99%

“…For ethnicity, we started by comparing each ethnic group to one another (e.g., Black/African American vs. Hispanic/Latinx students). Next, because ethnicity is a multicategory variable, we ran a multiple-group omnibus DIF (MG-DIF) in which the BG was the reference group (Oshima et al, 2015). Finally, to identify which groups showed DIF in the MG-DIF, we compared each focal ethnic group to the BG (e.g., Asian students vs. BG).…”

Section: Data Analysis Proceduresmentioning

confidence: 99%

“…In a typical DIF analysis, two groups, referred to as the reference and focal groups, are selected for comparison (Osterlind & Everson, 2010). From a statistical perspective, when comparing two groups, which group serves as the reference versus focal group is arbitrary (Ellis & Kimmel, 1992; Oshima et al, 2015). However, with a multicategory grouping variable such as ethnicity that lends itself to a multiple-group DIF analysis, the choice of reference group becomes more important (Ellis & Kimmel, 1992; Sari & Huggins, 2015).…”

Section: Literature Review Achievement Motivationmentioning

confidence: 99%

See 3 more Smart Citations

Fairness and Comparability in Achievement Motivation Items: A Differential Item Functioning Analysis

Bialo

2022

Journal of Psychoeducational Assessment

View full text Add to dashboard Cite

Achievement motivation is a well-documented predictor of a variety of positive student outcomes. However, given observed group differences in motivation and related outcomes, motivation instruments should be checked for comparable item and scale functioning. Therefore, the purpose of this study was to evaluate measurement scale comparability and differential item functioning (DIF) in PISA 2015 achievement motivation items across gender and ethnicity using pairwise and multiple-group comparisons. In addition, DIF was investigated in relation to a common base group that reflected the sample average. Results indicated DIF between gender groups and between the base group and female students. For ethnicity, DIF was consistently flagged in pairwise comparisons with Black/African American students and Asian students as well as in base group comparisons. However, the identified DIF had little practical implications. Implications from these findings are discussed, and recommendations for future research are made.

show abstract

Section: Measures and Variablesmentioning

confidence: 99%

Section: Measures and Variablesmentioning

confidence: 99%

Section: Data Analysis Proceduresmentioning

confidence: 99%

Section: Literature Review Achievement Motivationmentioning

confidence: 99%

See 2 more Smart Citations

Fairness and Comparability in Achievement Motivation Items: A Differential Item Functioning Analysis

Bialo

2022

Journal of Psychoeducational Assessment

View full text Add to dashboard Cite

show abstract

“…In the literature, various parametric and nonparametric statistical techniques and the associated effect-size measures detecting uniform DIF have been well documented. These are the Mantel–Haenszel (MH) procedure (Camilli & Shepard, 1994; Hidalgo et al, 2014; Holland & Thayer, 1988; Zwick et al, 2012), logistic regression (LR) modeling (Gómez-Benito et al, 2009; Hidalgo et al, 2014), item response theory (IRT)–based method (Oshima et al, 2015; Raju, 1988; Steinberg & Thissen, 2006), structural equation modeling (SEM; Bauer, 2017; Steinmetz et al, 2009; Woods & Grimm, 2011), and variations of the aforementioned techniques (Chang et al, 1995; Penfield, 2007; Walker, 2011).…”

mentioning

confidence: 99%

Differential Item Functioning Effect Size From the Multigroup Confirmatory Factor Analysis for a Meta-Analysis: A Simulation Study

Park

Ahn

Zopluoğlu

2020

Educational and Psychological Measurement

View full text Add to dashboard Cite

This study presents a new approach to synthesizing differential item functioning (DIF) effect size: First, using correlation matrices from each study, we perform a multigroup confirmatory factor analysis (MGCFA) that examines measurement invariance of a test item between two subgroups (i.e., focal and reference groups). Then we synthesize, across the studies, the differences in the estimated factor loadings between the two subgroups, resulting in a meta-analytic summary of the MGCFA effect sizes (MGCFA-ES). The performance of this new approach was examined using a Monte Carlo simulation, where we created 108 conditions by four factors: (1) three levels of item difficulty, (2) four magnitudes of DIF, (3) three levels of sample size, and (4) three types of correlation matrix (tetrachoric, adjusted Pearson, and Pearson). Results indicate that when MGCFA is fitted to tetrachoric correlation matrices, the meta-analytic summary of the MGCFA-ES performed best in terms of bias and mean square error values, 95% confidence interval coverages, empirical standard errors, Type I error rates, and statistical power; and reasonably well with adjusted Pearson correlation matrices. In addition, when tetrachoric correlation matrices are used, a meta-analytic summary of the MGCFA-ES performed well, particularly, under the condition that a high difficulty item with a large DIF was administered to a large sample size. Our result offers an option for synthesizing the magnitude of DIF on a flagged item across studies in practice.

show abstract

A Nonparametric Composite Group DIF Index for Focal Groups Stemming from Multicategorical Variables

Huggins‐Manley,

Raborn,

Jones

et al. 2024

J Educational Measurement

View full text Add to dashboard Cite

The purpose of this study is to develop a nonparametric DIF method that (a) compares focal groups directly to the composite group that will be used to develop the reported test score scale, and (b) allows practitioners to explore for DIF related to focal groups stemming from multicategorical variables that constitute a small proportion of the overall testing population. We propose the nonparametric root expected proportion squared difference (REPSD) index that evaluates the statistical significance of composite group DIF for relatively small focal groups stemming from multicategorical focal variables, with decisions of statistical significance based on quasi‐exact p values obtained from Monte Carlo permutations of the DIF statistic under the null distribution. We conduct a simulation to evaluate conditions under which the index produces acceptable Type I error and power rates, as well as an application to a school district assessment. Practitioners can calculate the REPSD index in a freely available package we created in the R environment.

show abstract

Multiple-Group Noncompensatory Differential Item Functioning in Raju's Differential Functioning of Items and Tests

Cited by 6 publications

References 18 publications

Fairness and Comparability in Achievement Motivation Items: A Differential Item Functioning Analysis

Fairness and Comparability in Achievement Motivation Items: A Differential Item Functioning Analysis

Differential Item Functioning Effect Size From the Multigroup Confirmatory Factor Analysis for a Meta-Analysis: A Simulation Study

A Nonparametric Composite Group DIF Index for Focal Groups Stemming from Multicategorical Variables

Contact Info

Product

Resources

About