Examining the Impact of Differential Item Functioning on Classification Accuracy in Cognitive Diagnostic Models

Paulsen, Justin; Svetina, Dubravka; Feng, Yanan; Valdivia, Montserrat

doi:10.1177/0146621619858675

Cited by 10 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to Paulsen et al (2020) and Qiu et al (2019), this study considered that 0%, 20%, and 40% of items exhibited DIF. The DIF items were randomly selected from all possible items with the constraint that one-third of the DIF items required a single attribute, one-third required two attributes, and the remaining one-third required three attributes.…”

Section: Simulation Studymentioning

confidence: 99%

“…Similarly, in the CDM context, an item is said to function differently when the probability of success to an item differs across manifest groups of students with the same attribute profile (Hou et al, 2014). The presence of DIF items has been viewed as a potential threat to test validity and could worsen the attribute estimation (Paulsen et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Li, 2008; X. Li & Wang, 2015; Liu et al, 2019; Paulsen et al, 2020; Zhang, 2006) investigated the DIF detection based on the DINA model, which is one of the simplest CDMs and may not hold in practice. An exception is X.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models

Terzi

Torre

2020

Applied Psychological Measurement

View full text Add to dashboard Cite

This study proposes a multiple-group cognitive diagnosis model to account for the fact that students in different groups may use distinct attributes or use the same attributes but in different manners (e.g., conjunctive, disjunctive, and compensatory) to solve problems. Based on the proposed model, this study systematically investigates the performance of the likelihood ratio (LR) test and Wald test in detecting differential item functioning (DIF). A forward anchor item search procedure was also proposed to identify a set of anchor items with invariant item parameters across groups. Results showed that the LR and Wald tests with the forward anchor item search algorithm produced better calibrated Type I error rates than the ordinary LR and Wald tests, especially when items were of low quality. A set of real data were also analyzed to illustrate the use of these DIF detection procedures.

show abstract

Section: Simulation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models

Terzi

Torre

2020

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…Because of the increase of data size and development of computational power, artificial neural networks (ANNs; Goodfellow et al, 2016 ) have been proposed as an attractive approach to convert a pattern of item responses into a diagnostic classification (Cui et al, 2016 ; Guo et al, 2017 ; Paulsen, 2019 ; Xue, 2019 ). An ANN is a computational system inspired by biological neural systems for information processing in animals' brains.…”

Section: Introductionmentioning

confidence: 99%

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks

Xue

Bradshaw

2021

Front. Psychol.

View full text Add to dashboard Cite

The purpose of cognitive diagnostic modeling (CDM) is to classify students' latent attribute profiles using their responses to the diagnostic assessment. In recent years, each diagnostic classification model (DCM) makes different assumptions about the relationship between a student's response pattern and attribute profile. The previous research studies showed that the inappropriate DCMs and inaccurate Q-matrix impact diagnostic classification accuracy. Artificial Neural Networks (ANNs) have been proposed as a promising approach to convert a pattern of item responses into a diagnostic classification in some research studies. However, the ANNs methods produced very unstable and unappreciated estimation unless a great deal of care was taken. In this research, we combined ANNs with two typical DCMs, the deterministic-input, noisy, “and” gate (DINA) model and the deterministic-inputs, noisy, “or” gate (DINO) model, within a semi-supervised learning framework to achieve a robust and accurate classification. In both simulated study and real data study, the experimental results showed that the proposed method could achieve appreciated performance across different test conditions, especially when the diagnostic quality of assessment was not high and the Q-matrix contained misspecified elements. This research study is the first time of applying the thinking of semi-supervised learning into CDM. Also, we used the validating test to choose the appropriate parameters for the ANNs instead of using typical statistical criteria.

show abstract

“…DIF analysis has been proficiently used in several psychological fields, such as social (e.g., [ 29 , 30 , 31 ]), clinical (e.g., [ 32 , 33 ]), and cognitive (e.g., [ 34 , 35 ]) psychology. Its use seems not to be confined to paper-and-pencil self-reported questionnaires only, but there also is a large employment for behavioral (e.g., [ 36 ]), neuropsychological and performance-based tools (e.g., [ 37 , 38 , 39 ]).…”

Section: Introductionmentioning

confidence: 99%

Studying Individual Differences in Spatial Cognition Through Differential Item Functioning Analysis

et al. 2020

View full text Add to dashboard Cite

Background: In the field of spatial cognition, the study of individual differences represents a typical research topic. Gender and age have been prominently investigated. A promising statistical technique used to identify the different responses to items in relation to different group memberships is the Differential Item Functioning Analysis (DIF). The aim of the present study was to investigate the DIF of the Landmark positioning on a Map (LPM) task, across age groups (young and elderly) and gender, in a sample of 400 healthy human participants. Methods: LPM is a hometown map completion test based on well-known and familiar landmarks used to assess allocentric mental representations. DIF was assessed on LPM items two times: on categorical (i.e., positions) and coordinate (i.e., distances) scores, separately. Results: When positions and distances were difficult to assess with respect to the intended reference point, the probability to endorse the items seemed to get worse for the elderly compared to the younger participants. Instead other features of landmarks (high pleasantness, restorativeness) seemed to improve the elderly performance. A gender-related improvement of probability to endorse distance estimation of some landmarks, favoring women, emerged, probably associated with their repeated experiences with those landmarks. Overall, the complexity of the task seemed to have a differential impact on young and elderly people while gender-oriented activities and places seemed to have a differential impact on men and women. Conclusions: For the first time DIF was applied to a spatial mental representation task, based on the schematic sketch maps of the participants. The application of DIF to the study of individual differences in spatial cognition should become a systematic routine to early detect differential items, improving knowledge, as well as experimental control, on individual differences.

show abstract

Examining the Impact of Differential Item Functioning on Classification Accuracy in Cognitive Diagnostic Models

Cited by 10 publications

References 22 publications

Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models

Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks

Studying Individual Differences in Spatial Cognition Through Differential Item Functioning Analysis

Contact Info

Product

Resources

About