Generalized Mantel-Haenszel Methods for Differential Item Functioning Detection

Fidalgo, Ángel M.; Madeira, Jaqueline M.

doi:10.1177/0013164408315265

Cited by 29 publications

(27 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Over the last several decades, the literature has featured many DIF methods suitable for two groups (one reference and one focal) and reviews of the methods suitable for dichotomous or polytomous items (e.g., Camilli & Shepard 1994;Millsap & Everson 1993;Penfield & Lam 2000;Potenza & Dorans 1995). Furthermore, performance of different DIF methods have been examined (e.g., Li et al 2012;Penfield 2001), including those that allow for multiple groups (i.e., > 2), such as generalized MantelHaenszel (e.g., Fidalgo & Madeira 2008;Fidalgo & Scalon 2010;Penfield 2001), generalized Lord's test (e.g., Kim et al 1995) and generalized logistic regression Magis et al 2011).…”

Section: Methods To Examine Item-level Difmentioning

confidence: 99%

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Svetina

Rutkowski

2014

Large-scale Assess Educ

View full text Add to dashboard Cite

Background: When studying student performance across different countries or cultures, an important aspect for comparisons is that of score comparability. In other words, it is imperative that the latent variable (i.e., construct of interest) is understood and measured equivalently across all participating groups or countries, if our inferences regarding performance can be regarded as valid. Relatively fewer studies examined an item-level approach to measurement equivalence, particularly in settings where a large number of groups is included. Methods: This simulation study examines item-level differential item functioning (DIF) in the context of international large-scale assessment (ILSA) using a generalized logistic regression approach. Manipulated factors included the number of groups (10 or 20), magnitude of DIF, percent of DIF items, the nature of DIF, as well as the percent of affected groups with DIF. Results: Results suggested that the number of groups did not have an effect of the performance of the method (high power and low Type I error rates); however, other factors had impacted the accuracy. Specifically, Type I error rates were inflated in non-DIF conditions, while they were very conservative in all of the DIF conditions. Power was generally high, in particular in conditions where DIF magnitude was large, with one exception -in conditions where DIF was introduced in difficulty parameters and the percent of DIF items was 60. Conclusions: Our findings presented a mixed picture with respect to the performance of the generalized logistic regression method in the context of large number of groups with large sample sizes. In the presence of DIF, the method was successful in distinguishing between DIF and non-DIF, as evidenced by low Type I error and high power rates. On the other hand, however, in the absence of DIF, the method yielded increased Type I errors.

show abstract

Section: Methods To Examine Item-level Difmentioning

confidence: 99%

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Svetina

Rutkowski

2014

Large-scale Assess Educ

View full text Add to dashboard Cite

show abstract

“…Studies have found that large mean differences in ability distributions across groups are associated with decreased power to detect DIF using non-IRT methods (e.g., Klockars & Lee, 2008;Mazor, Clauser, & Hambleton, 1992;Narayanan & Swaminathan, 1994, 1996. In addition, the MH statistic and its modifications have been shown to have higher Type I error rates as ability distributions become more discrepant and discrimination parameters differ across groups (Fidalgo & Madeira, 2008).…”

Section: Differences In Ability Distributionsmentioning

confidence: 99%

“…However, complexities still exist. For example, some DIF studies for students with disabilities have had reference and focal groups tested under different conditions and different proficiency distributions that can influence the proportion and type of items that are flagged for DIF (Dorans & Holland, 1993;Fidalgo & Madeira, 2008;Sireci, 2009). …”

Section: Dif and Students With Disabilitiesmentioning

confidence: 99%

“…We found a slight trend of higher percentages of items being flagged for DIF as the ability distributions between the groups became more discrepant. Future DIF studies on students with disabilities should take into account the methodological literature and employ methods that are most robust to discrepant ability distributions (e.g., Fidalgo & Madeira, 2008;Klockars & Lee, 2008;Mazor, Clauser, & Hambleton, 1992;Narayanan & Swaminathan, 1994, 1996 when the groups of interest are not expected to perform similarly on the total test due to their disability. In addition, we caution against combining students with different disability subtypes, particularly those who differ in their impact on cognitive ability, and instead support the creation of separable, well-defined focal and reference groups that are defined by theoretically important research questions rather than sampling convenience.…”

Section: Recommendations For Studying Differential Item Functioning (mentioning

confidence: 99%

See 1 more Smart Citation

Recommendations for Conducting Differential Item Functioning (Dif) Analyses for Students With Disabilities Based on Previous Dif Studies

Buzick

Stone

2011

ETS Research Report Series

View full text Add to dashboard Cite

The purpose of this study is to help ensure that strategies for differential item functioning (DIF) detection for students with disabilities are appropriate and lead to meaningful results. We surveyed existing DIF studies for students with disabilities and describe them in terms of study design, statistical approach, sample characteristics, and DIF results. Based on descriptive and graphical summaries of previous DIF studies, we make recommendations for future studies of DIF for students with disabilities.

show abstract

“…If items with differential func-tioning are included in the test, overall score of the test can favor one or another group and it can produce socially, legally or politically significant bias affecting the different groups (McDonald, 2013). To date, several statistical methods for detecting DIF have been applied, such as the Mantel-Haenszel procedure (Holland & Thayer, 1988;Fidalgo & Madeira, 2008), logistic regression (French & Miller, 1996;Zumbo, 1999) or item response theory approach (Jelínek, Květon, & Vobořil, 2011). There are two types of DIF: uniform and non-uniform.…”

mentioning

confidence: 99%

Gender Differential Item Functioning in Slovak Version of Intelligence Structure Test 2000 - Revised

Kohút

Haľama²,

Dočkal³

et al. 2016

View full text Add to dashboard Cite

Abstract:The study focused on the gender differential item functioning in Slovak version of the Intelligence Structure Test 2000 -Revised (Amthauer et al., 2011). The sample included 744 middle and high school students with mean age of 16.94 years. The non-parametric method SIBTEST for identification of items with differential functioning was used in order to detect uniform and non-uniform DIF. The analysis showed that the I-S-T 2000 R includes several items with DIF favoring either males or females, but in most subtests, with no or small effect on differences between genders. Substantial but nonsignificant effect of DIF items on subtest score was found for Verbal Analogy, which contained six items with DIF all favoring females. These items included verbal content related to areas more common for females such as diet or food. The results suggest that specific content of verbal intelligence items can be a potential source of gender bias.

show abstract

Generalized Mantel-Haenszel Methods for Differential Item Functioning Detection

Cited by 29 publications

References 65 publications

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Recommendations for Conducting Differential Item Functioning (Dif) Analyses for Students With Disabilities Based on Previous Dif Studies

Gender Differential Item Functioning in Slovak Version of Intelligence Structure Test 2000 - Revised

Contact Info

Product

Resources

About