A Binary Programming Approach to Automated Test Assembly for Cognitive Diagnosis Models

Finkelman, Matthew; Kim, Wonsuk; Roussos, Louis A.; Verschoor, Angela J.

doi:10.1177/0146621609344846

Cited by 16 publications

(16 citation statements)

References 22 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this issue, ADI was developed to indicate the discrimination power of an item with respect to each of the attributes. Second, genetic algorithm (GA; Finkelman, Kim, & Roussos, 2009) and binary programming (BP; Finkelman, Kim, Roussos, & Verschoor, 2010) have been proposed for test construction purposes in the context of CDMs. Finkelman et al (2010) provided guidelines on how to choose from different test construction methods, namely, CDI, BP, and GA.…”

Section: Introductionmentioning

confidence: 99%

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

Kuo

Pai

Torre

2016

Applied Psychological Measurement

View full text Add to dashboard Cite

At present, there are only a limited number of studies examining how to optimally construct cognitive diagnostic tests. The cognitive diagnostic index (CDI) and attribute-level discrimination index (ADI) have been proposed to assemble such tests. The CDI and ADI have been shown to be instrumental in constructing cognitive diagnostic tests when the attribute relationships are assumed to be nonhierarchical. For greater generality when designing cognitive diagnostic assessment, attribute hierarchy and the ratio of test length to the number of attributes (RTA) are two important factors to be considered. This article proposes modified indices that take into account attribute hierarchy and RTA. Simulation studies show that, under the deterministic input, noisy, "and" gate model (DINA) and the reduced version of the reparameterized unified model (rRUM), the proposed indices provide higher attribute and attribute pattern correct classification rates than the original indices.

show abstract

Section: Introductionmentioning

confidence: 99%

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

Kuo

Pai

Torre

2016

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…Henson et al (2008) developed the attribute discrimination index (ADI) to compute the information each attribute provided. Then Finkelman et al (2010) developed a binary programming method based on ADI to assemble tests automatically for CDM. ADI aims to compute the expected KL information between any two AMPs, with all the attributes holding constant except the target attribute, within the ideal response pattern (IRP; Tatsuoka, 1995).…”

Section: Balance Attribute Coverage Based On Attribute Discriminationmentioning

confidence: 99%

“…where ADI (A)k is the lower bound ADI of attribute k and the value of ADI (A)k is the average of ADI (A)k1 and ADI (A)k0 (Finkelman et al, 2010); adi (A)k represents ADI of attribute k that has already been selected. The difference between the number of items measuring each attribute-based (MGDI-based) ABI and ADI (A) -based ABI is that B k and b k are both positive integers and ABIs are nonnegative, whereas ADI (A)k and adi (A)k include any values that larger than 0.…”

Section: Balance Attribute Coverage Based On Attribute Discriminationmentioning

confidence: 99%

Attribute Discrimination Index-Based Method to Balance Attribute Coverage for Short-Length Cognitive Diagnostic Computerized Adaptive Testing

et al. 2020

View full text Add to dashboard Cite

We propose a new method that balances attribute coverage for short-length cognitive diagnostic computerized adaptive testing (CD-CAT). The new method uses the attribute discrimination index (ADI-based method) instead of the number of items that measure each attribute [modified global discrimination index (MGDI)-based method] to balance the attribute coverage. Therefore, the information that each attribute provides can be captured. The purpose of the simulation study was to evaluate the performance of the new method, and the results showed the following: (a) Compared with uncontrolled attribute-balance coverage method, the new method produced a higher mastery pattern correct classification rate (PCCR) and attribute correct classification rate (ACCR) with both the posterior-weighted Kullback-Leibler (PWKL) and the modified PWKL (MPWKL) item selection method. (b) Equalization of ACCR (E-ACCR) based on the ADI-based method leads to better results, followed by the MGDI-based method. The uncontrolled method leads to the worst results regardless of item selection methods. (c) Both the ADI-based and MGDI-based methods produced acceptable examinee qualification rates, regardless of item selection methods, although they were relatively low for the uncontrolled condition.Keywords: balance attribute coverage, cognitive diagnostic computerized adaptive testing, attribute discrimination index, equalization of attribute correct classification rate, examinee qualification rate Frontiers in Psychology | www.frontiersin.org

show abstract

“…Drawing on previous studies (Cheng & Chang, ; Finkelman, Kim, Roussos, & Verschoor, ; Mao & Xin, ), the nonstatistical constraints in this study (see Table ) are set as (1) each attribute should be measured at least three times for the whole test; (2) the bank consists of items from five content areas, and the minimum number of items from each content area is set at two; (3) there are two different question types, and no less than seven items are adopted for each type; (4) no less than three items with correct answer key A, B, C, or D, respectively, are required; and (5) two items are selected randomly as conflicting items, which means the selection of one conflicting item excluded the selection of the other one, as one conflicting item provides a clue to the solution of the other conflicting item (Finkelman et al, ).…”

Section: Simulation Studymentioning

confidence: 99%

On‐the‐Fly Constraint‐Controlled Assembly Methods for Multistage Adaptive Testing for Cognitive Diagnosis

Liu

Cai

2018

J Educational Measurement

View full text Add to dashboard Cite

This study applied the mode of on‐the‐fly assembled multistage adaptive testing to cognitive diagnosis (CD‐OMST). Several and several module assembly methods for CD‐OMST were proposed and compared in terms of measurement precision, test security, and constrain management. The module assembly methods in the study included the maximum priority index method (MPI), the revised maximum priority index (RMPI), the weighted deviation model (WDM), and the two revised Monte Carlo methods (R1‐MC, R2‐MC). Simulation results showed that on the whole the CD‐OMST performs well in that it not only has acceptable attribute pattern correct classification rates but also satisfies both statistical and nonstatistical constraints; the RMPI method was generally better than the MPI method, the R2‐MC method was generally better than the R1‐MC method, and the two revised Monte Carlo methods performed best in terms of test security and constraint management, whereas the RMPI and WDM methods worked best in terms of measurement precision. The study is not only expected to provide information about how to combine MST and CD using an on‐the‐fly method and how do these assembled methods in CD‐OMST perform relative to each other but also offer guidance for practitioners to assemble modules in CD‐OMST with both statistical and nonstatistical constraints.

show abstract

A Binary Programming Approach to Automated Test Assembly for Cognitive Diagnosis Models

Cited by 16 publications

References 22 publications

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

Attribute Discrimination Index-Based Method to Balance Attribute Coverage for Short-Length Cognitive Diagnostic Computerized Adaptive Testing

On‐the‐Fly Constraint‐Controlled Assembly Methods for Multistage Adaptive Testing for Cognitive Diagnosis

Contact Info

Product

Resources

About