Item Selection for the Development of Parallel Forms From an IRT-Based Seed Test Using a Sampling and Classification Approach

Abstract

Two sampling-and-classification–based procedures were developed for automated test assembly: the Cell Only and the Cell and Cube methods. A simulation study based on a 540-item bank was conducted to compare the performance of the procedures with the performance of a mixed-integer programming (MIP) method for assembling multiple parallel test forms. The study investigated the statistical equivalence of the forms generated by the three test assembly methods (Cell Only, Cell and Cube, and MIP) in terms of test information functions, test characteristic curves, mean square deviations, and practical constraints, such as content balancing and nonoverlap among forms. The results indicated that the 13-point MIP method outperformed the other two methods in terms of the “closeness” test information functions between the reference form and the assembled parallel tests. Regarding test characteristic curves, the Cell Only and Cell and Cube methods yielded more similar test characteristic curves than the MIP method. Constraining test information functions apparently does not guarantee that the assembled forms will yield similar test characteristic curves. Overall, the Cell Only and Cell and Cube methods have the potential to provide results similar to the optimization approach.

Variable neighborhood search heuristics for a test assembly design problem

“…These results apply under varying conditions, such as when the items are grouped according to similar weights, as in Brusco et al (2013), or when the items are randomly grouped. Results for randomly grouped items consider cases in which additional constraints are to be enforced (He et al, 2014;Ishii et al, 2014;Nguyen & Fong, 2013), or cases in which a cell approach (Chen et al, 2012;Chen, 2015) is used to maintain similar characteristics not only among questionnaires, but also among the item composition of each questionnaire.…”

“…Chen et al (2012) proposes one of these sequential methods. The method first divides the item pool into groups (referred as cells) containing questions with similar characteristics.…”

“…The authors consider that the item pool has been previously divided into groups of items with similar characteristics, as seen in van der Linden & Boekkooi-Timminga (1988), Chen et al (2012) or Chen (2015), and then they consider the generation of multiple tests each containing an item from each group.…”

Evaluating Statistical Targets for Assembling Parallel Mixed‐Format Test Forms

“…We expect that the Minimax models with only the TIF or only the TCC as statistical target will have a good overall performance for an item pool with only 1PLM items (i.e., condition I). In item pools with 2PLM and 3PLM items (i.e., condition II and III) we expect satisfactory performance with respect to the TIF but not with respect to the TCC for the Only TIF Minimax model, while the opposite is expected for the Only TCC Minimax model (Ali and van Rijn, ; Chen et al., ). Further, a similar pattern is expected for the mixed‐format item pools, regardless the used item response models (i.e., in condition IV through IX).…”

“…Different methods are available. Some are based on sampling-and-classification methods such as the Cell Only and the Cell and Cube methods (Chen et al, 2012), while others rely on constrained combinatoral optimization techniques (e.g., Finkelman, Kim, and Roussos, 2009;Luecht, 1998;van der Linden, 2005). The most commonly used optimization technique involves translating the ATA problem to a mixed integer linear programming (MILP) model (Diao and van der Linden, 2011;Theunissen, 1985;van der Linden, 2005).…”