An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix

Liu, Yanlou; Tao, Xin; Lingqing, LI; Wei, Tian; Liu, Xiaoxiao

doi:10.3724/sp.j.1041.2016.00588

Cited by 6 publications

(17 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under the null hypothesis of H 0 : Cv j = 0 , W d asymptotically follows a chi-square distribution with 2 degrees of freedom. However, authors of previous studies (Hou et al, 2014; Svetina et al, 2018) showed that W d tended to have inflated Type I errors and the W XPD and W Obs performed better than that for W d with regard to Type I error control (Liu et al, 2016b).…”

Section: Introductionmentioning

confidence: 96%

“…Several DIF detection methods have been proposed and investigated in the framework of CDMs (Zhang, 2006; Li, 2008; Hou et al, 2014; Wang et al, 2014; Li and Wang, 2015; Liu et al, 2016b), which can be classified into two types, CDM based and not. For example, the modified higher-order DINA (Li, 2008) and log-linear cognitive diagnosis models for DIF assessment (Li and Wang, 2015), for which the model parameters were estimated using the Markov chain Monte Carlo (MCMC) algorithm, and the Wald test, for which MCMC or maximum likelihood estimation was used for estimating the item parameters (Hou et al, 2014; Li and Wang, 2015) are the CDM-based method.…”

Section: Introductionmentioning

confidence: 99%

“…The item-wise information matrix underestimate the variance-covariance matrix of item parameters (Philipp et al, 2018). As alternatives, cross-product (XPD) information, observed (Obs) information, and sandwich-type (Sw) covariance matrices have been proposed for estimating asymptotic covariance matrices of item parameters (Liu et al, 2016b, 2019; Philipp et al, 2018). That is, the item parameter covariance matrix used to compute the Wald statistic can be estimated using XPD, Obs, or Sw matrix (statistics denoted as W XPD , W Obs , and W Sw , respectively).…”

Section: Introductionmentioning

confidence: 99%

“…That is, the item parameter covariance matrix used to compute the Wald statistic can be estimated using XPD, Obs, or Sw matrix (statistics denoted as W XPD , W Obs , and W Sw , respectively). Liu et al (2016b) evaluated the empirical performance of W XPD and W Obs for detecting DIF in CDMs following Hou et al's (2014) simulation design. They found that when the sample size was 1,000 and the attribute correlation was 0, following Bradley's (1978) liberal criteria (1978), W XPD and W Obs had accurate Type I error rates.…”

Section: Introductionmentioning

confidence: 99%

“…Although the maximum marginal likelihood with the expectation–maximization (EM) algorithm provides an elegant solution to estimating the model parameters of CDMs, computing the variance-covariance matrix for item parameters is a challenging process under the EM framework. Calculating the matrix requires the inverse of the information matrix in which the item and structural parameters should be simultaneously considered (Liu et al, 2016b). Denote the marginal likelihood of the n th examinee's response pattern x n as…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

et al. 2019

Self Cite

View full text Add to dashboard Cite

As a class of discrete latent variable models, cognitive diagnostic models have been widely researched in education, psychology, and many other disciplines. Detecting and eliminating differential item functioning (DIF) items from cognitive diagnostic tests is of great importance for test fairness and validity. A Monte Carlo study with varying manipulated factors was carried out to investigate the performance of the Mantel-Haenszel (MH), logistic regression (LR), and Wald tests based on item-wise information, cross-product information, observed information, and sandwich-type covariance matrices (denoted by W d , W XPD , W Obs , and W Sw , respectively) for DIF detection. The results showed that (1) the W XPD and LR methods had the best performance in controlling Type I error rates among the six methods investigated in this study and (2) under the uniform DIF condition, when the item quality was high or medium, the power of W XPD , W Obs , and W Sw was comparable with or superior to that of MH and LR, but when the item quality was low, W XPD , W Obs , and W Sw were less powerful than MH and LR. Under the non-uniform DIF condition, the power of W XPD , W Obs , and W Sw was comparable with or higher than that of LR.

show abstract

Section: Introductionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

Evaluating the Fit of Sequential G-DINA Model Using Limited-Information Measures

2019

Applied Psychological Measurement

View full text Add to dashboard Cite

Limited-information fit measures appear to be promising in assessing the goodness-of-fit of dichotomous response cognitive diagnosis models (CDMs), but their performance has not been examined for polytomous response CDMs. This study investigates the performance of the Mord statistic and standardized root mean square residual (SRMSR) for an ordinal response CDM—the sequential generalized deterministic inputs, noisy “and” gate model. Simulation studies showed that the Mord statistic had well-calibrated Type I error rates, but the correct detection rates were influenced by various factors such as item quality, sample size, and the number of response categories. In addition, the SRMSR was also influenced by many factors and the common practice of comparing the SRMSR against a prespecified cut-off (e.g., .05) may not be appropriate. A set of real data was analyzed as well to illustrate the use of Mord statistic and SRMSR in practice.

show abstract

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests

Sun

Wang

Guo

et al. 2023

Applied Psychological Measurement

Self Cite

View full text Add to dashboard Cite

Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.

show abstract

An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix

Cited by 6 publications

References 0 publications

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

Evaluating the Fit of Sequential G-DINA Model Using Limited-Information Measures

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests

Contact Info

Product

Resources

About