Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment

Abstract

A variety of cognitive diagnostic models (CDMs) have been developed in recent years to help with the diagnostic assessment and evaluation of students. Each model makes different assumptions about the relationship between students’ achievement and skills, which makes it important to empirically investigate which CDMs better fit the actual data. In this study, we examined this question by comparatively fitting representative CDMs to the Trends in International Mathematics and Science Study (TIMSS) 2007 assessment data across seven countries. The following two major findings emerged. First, in accordance with former studies, CDMs had a better fit than did the item response theory models. Second, main effects models generally had a better fit than other parsimonious or the saturated models. Related to the second finding, the fit of the traditional parsimonious models such as the DINA and DINO models were not optimal. The empirical educational implications of these findings are discussed.

Variational Bayes Inference for the DINA Model

“…Chen & de la Torre, 2014) and the Trends in International Mathematics and Science Study (K. K. Tatsuoka et al, 2004; Yamaguchi & Okada, 2018). Demand for large-scale application or real-time online estimation of CDMs can be expected to increase in the coming years.…”

“…Li, 2011), mathematics tests (e.g., K. K. Tatsuoka et al, 2004; Yamaguchi & Okada, 2018), and psychological tests (e.g., Sorrel et al, 2016; Templin & Henson, 2006).…”

“…From a modern point of view, this is surely a restrictive class of model compared to other later-proposed CDMs. Empirical model comparison studies (e.g., Yamaguchi & Okada, 2018) suggest that later-proposed, more richly parameterized CDMs tend to fit better than the DINA model. However, the DINA model is still a popular and important model with many applications.…”

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks

“…For example, when the attributes measured by an assessment are non-compensatory, which indicates that non-mastery on one attribute cannot be compensated by mastery on another attribute, selecting a compensatory model will decrease the performance of classification and measurement. DINA model and DINO (Templin and Henson, 2006 ) model achieved worse fit than did the other more relaxed DCMs, such as G-DINA (DeCarlo, 2011 ), LCDM, and RUM because both DINA and DINO might be too restrictive to reflect actual students' knowledge status (Yamaguchi and Okada, 2018 ). Some recent research studies (Chiu and Köhn, 2019 ; Yamaguchi and Okada, 2020 ; Zhan, 2020 ) started to apply the non-compensatory or conjunctive DCM, DINA model, and the compensatory or disjunctive DCM, DINO model, to build up a more general item response function (IRF) for CDM.…”

A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models

“…The equivalency of the three models is provided by de la Torre (2011) and von Davier (2014). If all possible parameters are included in these models they are also called saturated DCMs (Li, Hunter, & Lei, 2016;Yamaguchi & Okada, 2018), although this term has also been used to describe the DCM structural model (e.g., the model specifying the distributional parameters of the attributes, see Templin & Bradshaw, 2014, or Hu & Templin, 2019.…”