Combining regularization and logistic regression model to validate the Q‐matrix for cognitive diagnosis model

Abstract

Q‐matrix is an important component of most cognitive diagnosis models (CDMs); however, it mainly relies on subject matter experts' judgements in empirical studies, which introduces the possibility of misspecified q‐entries. To address this, statistical Q‐matrix validation methods have been proposed to aid experts' judgement. A few of these methods, including the multiple logistic regression‐based (MLR‐B) method and the Hull method, can be applied to general CDMs, but they are either time‐consuming or lack accuracy under certain conditions. In this study, we combine the L1 regularization and MLR model to validate the Q‐matrix. Specifically, an L1 penalty term is imposed on the log‐likelihood of the MLR model to select the necessary attributes for each item. A simulation study with various factors was conducted to examine the performance of the new method against the two existing methods. The results show that the regularized MLR‐B method (a) produces the highest Q‐matrix recovery rate (QRR) and true positive rate (TPR) for most conditions, especially with a small sample size; (b) yields a slightly higher true negative rate (TNR) than either the MLR‐B or the Hull method for most conditions; and (c) requires less computation time than the MLR‐B method and similar computation time as the Hull method. A real data set is analysed for illustration purposes.