Cognitive diagnosis models for multiple strategies

Abstract

Cognitive diagnosis models (CDMs) have been used as psychometric tools in educational assessments to estimate students' proficiency profiles. However, most CDMs assume that all students adopt the same strategy when approaching problems in an assessment, which may not be the case in practice. This study develops a generalized multiple-strategy CDM for dichotomous response data. The proposed model provides a unified framework to accommodate various condensation rules (e.g., conjunctive, disjunctive, and additive) and different strategy selection approaches (i.e., probability-matching, over-matching, and maximizing). Model parameters are estimated using the marginal maximum likelihood estimation via expectation-maximization algorithm. Simulation studies showed that the parameters of the proposed model can be adequately recovered and that the proposed model was relatively robust to some types of model misspecifications. A set of real data was analysed as well to illustrate the use of the proposed model in practice.

Estimating Cognitive Diagnosis Models in Small Samples: Bayes Modal Estimation and Monotonic Constraints

“…It can be expected that the nonconvergence issue would be more severe when sample size is small. Also, the EM algorithm has been observed to produce boundary or near-boundary solutions (e.g., Ma & Guo, 2019). Chiu et al (2018) noted that the EM algorithm may fail to “produce reasonable parameter estimates when samples are small” as in their real data analysis many estimates of success probabilities were either 0 or 1.…”

“…Furthermore, the boundary solutions pose a challenge to inferences. For example, obtaining standard errors of boundary solutions could be challenging or even impossible, as noted by Ma and Guo (2019) and Philipp et al (2018), and in turn, affects various hypothesis testing procedures for model comparison, differential item functioning detection and Q-matrix validation using the Wald and score tests (e.g., Ma & de la Torre, 2019; Sorrel et al, 2017).…”

GDINA: An R Package for Cognitive Diagnosis Modeling

“…These models can also be estimated using the GDINA() function. Second, the package can also calibrate various models for multiple strategies, including the multiple strategy DINA model (Huo and de la Torre 2014), generalized multiple strategy CDMs (Ma and Guo 2019), and diagnostic tree models (Ma 2019b). Other models and approaches for cognitive diagnosis that can be handled by the package include the multiple-choice DINA model (de la Torre 2009) and iterative latent class analysis (Jiang 2019).…”

“…The GDINA package also allows users to assess global model-data fit using the M 2 statistic for dichotomous response (Hansen, Cai, Monroe, and Li 2016;Liu, Tian, and Xin 2016) and the M ord statistic for ordinal response (Ma 2020). In addition, the GDINA package allows users to calibrate the generalized multiple-strategy models for dichotomous response (Ma and Guo 2019) and diagnostic tree model for polytomous response (Ma 2019b) when multiple strategies exist. Furthermore, the GDINA package provides a routine to validate the Q-matrix, choose the most appropriate CDMs for each item, and calibrate the selected CDMs at one fell swoop.…”

Improving Robustness in Q-Matrix Validation Using an Iterative and Dynamic Procedure

“…They should reject those suggested modifications that lack a theoretical interpretation and check that the attributes maintain their original meaning. Also, if they consider that several strategies can be followed to answer the items, multiple-strategy models may be of help (e.g., de la Torre & Douglas, 2008; Ma & Guo, 2019). These considerations may provide the most useful Q-matrix specification, since a trade-off between theoretical interpretation and data fit can be more easily achieved.…”