Item Discrimination and Type I Error in the Detection of Differential Item Functioning

Li, Yanju; Brooks, Gordon P.; Johanson, George A.

doi:10.1177/0013164411432333

Cited by 16 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…(1) in some studies, the LR method gave the same results as the MH method (Ankenmann, Witt, & Dunbar, 1996;DeMars, 2009;Vaughn & Wang, 2010), and (2) the error rate of the LR method was very high, and its statistical power, lower (Dainis, 2008;Hidalgo & Lopez-Pina, 2004;Jodoin & Gierl, 2001;Li, Brooks, & Johanson, 2012). In addition to this, the main weakness of the LR method in DIF determination is a tendency to produce higher Type I error (Li & Stout, 1996;Narayanan & Swaminathan, 1996;Rogers & Swaminathan, 1993;Swaminathan & Rogers, 1990).…”

Section: Dif Detection Methodsmentioning

confidence: 97%

“…An appropriate replication number is required for consistent results. In the literature, most researchers used 100 replications (Ankenmann et al, 1996;Dainis, 2008;DeMars, 2009;Erdem Keklik, 2012;Fukuhara, 2009;Kim, 2010;Narayanan & Swaminathan, 1996;Rogers & Swaminathan, 1993), but many research studies have used more than 100 replications (Finch & French, 2007;Güler & Penfiled, 2009;Li, 2012;Li et al, 2012;Uttara & Millsap, 1994). According to Diaz-Emparanza (1996), the replication number should be as high as possible to reduce error.…”

Section: Data Generationmentioning

confidence: 99%

See 1 more Smart Citation

Comparing Performances (Type I error and Power) of IRT Likelihood Ratio SIBTEST and Mantel-Haenszel Methods in the Determination of Differential Item Functioning

Kabasakal¹,

Gök²,

Arsan³

2014

EDUC SCI-THEOR PRACT

View full text Add to dashboard Cite

This simulation study compared the performances (Type I error and power) of Mantel-Haenszel (MH), SIBTEST, and item response theory-likelihood ratio (IRT-LR) methods under certain conditions. Manipulated factors were sample size, ability differences between groups, test length, the percentage of differential item functioning (DIF), and underlying model used to generate data. Results suggest that SIBTEST had the highest Type I error in the detection of uniform DIF, but MH had the highest power under all conditions. In addition, the percentage of DIF and the underlying model appear to have influenced the Type I error rate of IRT-LR. Ability differences between groups, test length, the percentage of DIF, model, and the interactions between ability differences*percentage of DIF, ability differences*test length, test length*percentage of DIF, test length*model affected the SIBTEST methods' Type I error rate. In the MH procedure, effective factors for Type I error rate were: sample size, test length, the percentage of DIF, ability differences*percentage of DIF, ability differences*model, and ability differences*percentage of DIF*model. No factors were effective on the power of SIBTEST and MH, but the underlying model had a significant effect on the IRT-LR power rate.

show abstract

Section: Dif Detection Methodsmentioning

confidence: 97%

Section: Data Generationmentioning

confidence: 99%

Comparing Performances (Type I error and Power) of IRT Likelihood Ratio SIBTEST and Mantel-Haenszel Methods in the Determination of Differential Item Functioning

Kabasakal¹,

Gök²,

Arsan³

2014

EDUC SCI-THEOR PRACT

View full text Add to dashboard Cite

show abstract

“…Sin embargo, nuestros resultados podrían ser razonablemente satisfactorios respecto a la invarianza de los parámetros analizados, más aún cuando otras características ocurrieron en los datos. Por ejemplo, la moderada similaridad de las cargas factoriales en los datos tiende a ser un protector contra el error Tipo I (Li, Brooks, & Johanson, 2012), especialmente entre cargas entre .60 y .90. En segundo lugar, los índices de ajuste alcanzaron un modesto ajuste -cerca, pero debajo de .95- (Bentler, & Wu, 2012;Tabachnick, & Fidell, 2007), pero pueden representar resultados robustos dado su aceptable desempeño en muestras pequeñas -por ejemplo, el cfi- (Hu, & Bentler, 1999).…”

Section: Discussionunclassified

“…38(1) / pp. 135-156 / 2020 / ISSNe2145-4515 latente de los ítems, lo cual ayuda a disminuir la inflación del error Tipo I (Li, Brooks, & Johanson, 2012), se verificó la estructura interna del mbi-gs en cada país antes de pasar al examen del dif entre países. Cabe aclarar que, aunque la escala no es unidimensional, el enfoque de análisis del supuesto se refirió a la unidimensionalidad de los ítems respecto al constructo de cada subescala.…”

Section: Estrategia De Análisis Psicométricosunclassified

Validación transcultural y funcionamiento diferencial del MBI-GS en docentes de tres países latinoamericanos

García

Merino-Soto

Fernández-Arata

et al. 2020

Av. Psicol. Latinoam.

View full text Add to dashboard Cite

En América Latina, la investigación del síndrome de burnout (SB) es creciente; sin embargo, poco se ha explorado sobre las posibles diferencias culturales en sus mecanismos psicológicos y eficiencia de medición. En particular, el cuestionario MBI-GS ha sido reconocido como una medida satisfactoria del SB para su uso en distintos sectores y grupos ocupacionales, y su validez ha sido probada en múltiples países e idiomas, empero en Latinoamérica se adolece de investigaciones al respecto. El objetivo de este estudio fue realizar una traducción y adaptación cultural del MBI-GS para población latinoamericana, analizar su estructura interna, su fiabilidad y su invarianza (DIF y MGFCA) entre sexos y muestras de tres países latinoamericanos: México, Perú y Venezuela. Los resultados evidencian que la versión latina del MBI-GS propuesta presenta propiedades psicométricas satisfactorias, incluyendo la ausencia del funcionamiento diferencial entre sexos y países participantes.

show abstract

“…Over the last several decades, the literature has featured many DIF methods suitable for two groups (one reference and one focal) and reviews of the methods suitable for dichotomous or polytomous items (e.g., Camilli & Shepard 1994;Millsap & Everson 1993;Penfield & Lam 2000;Potenza & Dorans 1995). Furthermore, performance of different DIF methods have been examined (e.g., Li et al 2012;Penfield 2001), including those that allow for multiple groups (i.e., > 2), such as generalized MantelHaenszel (e.g., Fidalgo & Madeira 2008;Fidalgo & Scalon 2010;Penfield 2001), generalized Lord's test (e.g., Kim et al 1995) and generalized logistic regression Magis et al 2011).…”

Section: Methods To Examine Item-level Difmentioning

confidence: 99%

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Svetina

Rutkowski

2014

Large-scale Assess Educ

View full text Add to dashboard Cite

Background: When studying student performance across different countries or cultures, an important aspect for comparisons is that of score comparability. In other words, it is imperative that the latent variable (i.e., construct of interest) is understood and measured equivalently across all participating groups or countries, if our inferences regarding performance can be regarded as valid. Relatively fewer studies examined an item-level approach to measurement equivalence, particularly in settings where a large number of groups is included. Methods: This simulation study examines item-level differential item functioning (DIF) in the context of international large-scale assessment (ILSA) using a generalized logistic regression approach. Manipulated factors included the number of groups (10 or 20), magnitude of DIF, percent of DIF items, the nature of DIF, as well as the percent of affected groups with DIF. Results: Results suggested that the number of groups did not have an effect of the performance of the method (high power and low Type I error rates); however, other factors had impacted the accuracy. Specifically, Type I error rates were inflated in non-DIF conditions, while they were very conservative in all of the DIF conditions. Power was generally high, in particular in conditions where DIF magnitude was large, with one exception -in conditions where DIF was introduced in difficulty parameters and the percent of DIF items was 60. Conclusions: Our findings presented a mixed picture with respect to the performance of the generalized logistic regression method in the context of large number of groups with large sample sizes. In the presence of DIF, the method was successful in distinguishing between DIF and non-DIF, as evidenced by low Type I error and high power rates. On the other hand, however, in the absence of DIF, the method yielded increased Type I errors.

show abstract

Item Discrimination and Type I Error in the Detection of Differential Item Functioning

Cited by 16 publications

References 21 publications

Comparing Performances (Type I error and Power) of IRT Likelihood Ratio SIBTEST and Mantel-Haenszel Methods in the Determination of Differential Item Functioning

Comparing Performances (Type I error and Power) of IRT Likelihood Ratio SIBTEST and Mantel-Haenszel Methods in the Determination of Differential Item Functioning

Validación transcultural y funcionamiento diferencial del MBI-GS en docentes de tres países latinoamericanos

Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments

Contact Info

Product

Resources

About