Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing

Yang, Jing; Chang, Hua Hua; Tao, Jiangchuan; Shi, Ning

doi:10.1177/0146621619893783

Cited by 4 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To ensure measurement accuracy, a test measuring more attributes commonly needs more items. Thus, the short test and long test in this study were set to 2 K and 4 K , respectively, which is in line with previous studies (e.g., Chang et al, 2019; Yang et al, 2020; Zheng & Chang, 2016). As a result, a complete crossed design for each simulation study was set as follows: five item selection methods (PWKL, SHE, GDI, HO-SHE, HO-GDI) × three higher-order models (HO-DINA, HO-RRUM, HO-GDINA) × two attribute numbers (6, 8) × two test lengths (short, long) = 60 conditions.…”

Section: Simulation Studymentioning

confidence: 65%

Dual-Objective Item Selection Methods in Computerized Adaptive Test Using the Higher-Order Cognitive Diagnostic Models

Cai

2022

Applied Psychological Measurement

View full text Add to dashboard Cite

To efficiently obtain information about both the general abilities and detailed cognitive profiles of examinees from a single model that uses a single-calibration process, higher-order cognitive diagnostic computerized adaptive testing (CD-CAT) that employ higher-order cognitive diagnostic models have been developed. However, the current item selection methods used in higher-order CD-CAT adaptively select items according to only the attribute profiles, which might lead to low precision regarding general abilities; hence, an appropriate method was proposed for this CAT system in this study. Under the framework of the higher-order models, the responses were affected by attribute profiles, which were governed by general abilities. It is reasonable to hold that the item responses were affected by a combination of general abilities and attribute profiles. Based on the logic of Shannon entropy and the generalized deterministic, inputs, noisy “and” gate (G-DINA) model discrimination index (GDI), two new item selection methods were proposed for higher-order CD-CAT by considering the above combination in this study. The simulation results demonstrated that the new methods achieved more accurate estimations of both general abilities and cognitive profiles than the existing methods and maintained distinct advantages in terms of item pool usage.

show abstract

Section: Simulation Studymentioning

confidence: 65%

Dual-Objective Item Selection Methods in Computerized Adaptive Test Using the Higher-Order Cognitive Diagnostic Models

Cai

2022

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…The core of CD‐CAT is the item selection method, which has yielded many fruitful achievements. Furthermore, research on the item selection method pays attention to two aspects: (1) one is to pursue high classification accuracies, such as the Shannon entropy (SHE) method (Xu et al, 2003), posterior‐weighted Kullback–Leibler information (PWKL) method (Cheng, 2009), mutual information (MI) method (Wang, 2013), modified PWKL and G‐DINA model discrimination index (GDI) method (Kaplan et al, 2015), methods based on the CDM discrimination index or the CDI (Henson & Douglas, 2005; Zheng & Chang, 2016) and methods for attribute balancing (Cheng, 2010; Wang et al, 2020); (2) the other is to maintain high classification accuracy and give consideration to the utilization rate of the item pool; this is represented by the attribute balancing design method (Lin & Chang, 2019), stratified‐design method (Yang et al, 2020), blocked‐design method (Kaplan & de la Torre, 2020) and binary searching design method (Zheng & Wang, 2017). However, most research explored the selection algorithm in low‐dimensional scenarios and seldom considered high‐dimensional scenarios, in which the number of attributes is large.…”

Section: Introductionmentioning

confidence: 99%

Two efficient selection methods for high‐dimensional CD‐CAT utilizing max‐marginals factor from MAP query and ensemble learning approach

Luo

Wang

Cai

et al. 2022

Brit J Math & Statis

View full text Add to dashboard Cite

Computerized adaptive testing for cognitive diagnosis (CD-CAT) needs to be efficient and responsive in real time to meet practical applications' requirements. For high-dimensional data, the number of categories to be recognized in a test grows exponentially as the number of attributes increases, which can easily cause system reaction time to be too long such that it adversely affects the examinees and thus seriously impacts the measurement efficiency. More importantly, the long-time CPU operations and memory usage of item selection in CD-CAT due to intensive computation are impractical and cannot wholly meet practice needs. This paper

show abstract

“…And the study showed that the performance of these two methods is better than the PWKL method when the calibration samples are small. In addition, Yang et al ( 2020 ) proposed three stratified item selection methods based on PWKL, NPS, and WNPS, named S-PWKL, S-NPS, and S-WNPS, respectively. Among them, the S-WNPS and S-NPS methods performed similarly and both of them are better than the S-PWKL method.…”

Section: Introductionmentioning

confidence: 99%

An Adaptive Design for Item Parameter Online Estimation and Q-Matrix Online Calibration in CD-CAT

Wang

Song

et al. 2021

Front. Psychol.

View full text Add to dashboard Cite

The implementation of cognitive diagnostic computerized adaptive testing often depends on a high-quality item bank. How to online estimate the item parameters and calibrate the Q-matrix required by items becomes an important problem in the construction of the high-quality item bank for personalized adaptive learning. The related previous research mainly focused on the calibration method with the random design in which the new items were randomly assigned to examinees. Although the way of randomly assigning new items can ensure the randomness of data sampling, some examinees cannot provide enough information about item parameter estimation or Q-matrix calibration for the new items. In order to increase design efficiency, we investigated three adaptive designs under different practical situations: (a) because the non-parametric classification method needs calibrated item attribute vectors, but not item parameters, the first study focused on an optimal design for the calibration of the Q-matrix of the new items based on Shannon entropy; (b) if the Q-matrix of the new items was specified by subject experts, an optimal design was designed for the estimation of item parameters based on Fisher information; and (c) if the Q-matrix and item parameters are unknown for the new items, we developed a hybrid optimal design for simultaneously estimating them. The simulation results showed that, the adaptive designs are better than the random design with a limited number of examinees in terms of the correct recovery rate of attribute vectors and the precision of item parameters.

show abstract

Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing

Cited by 4 publications

References 36 publications

Dual-Objective Item Selection Methods in Computerized Adaptive Test Using the Higher-Order Cognitive Diagnostic Models

Dual-Objective Item Selection Methods in Computerized Adaptive Test Using the Higher-Order Cognitive Diagnostic Models

Two efficient selection methods for high‐dimensional CD‐CAT utilizing max‐marginals factor from MAP query and ensemble learning approach

An Adaptive Design for Item Parameter Online Estimation and Q-Matrix Online Calibration in CD-CAT

Contact Info

Product

Resources

About