Look‐ahead content balancing method in variable‐length computerized classification testing

Abstract

Content balancing is one of the most important issues in computerized classification testing. To adapt to variable-length forms, special treatments are needed to successfully control content constraints without knowledge of test length during the test. To this end, we propose the notions of 'look-ahead' and 'step size' to adaptively control content constraints in each item selection step. The step size gives a prediction of the number of items to be selected at the current stage, that is, how far we will look ahead. Two look-ahead content balancing (LA-CB) methods, one with a constant step size and another with an adaptive step size, are proposed as feasible solutions to balancing content areas in variable-length computerized classification testing. The proposed LA-CB methods are compared with conventional item selection methods in variable-length tests and are examined with different classification methods. Simulation results show that, integrated with heuristic item selection methods, the proposed LA-CB methods result in fewer constraint violations and can maintain higher classification accuracy. In addition, the LA-CB method with an adaptive step size outperforms that with a constant step size in content management. Furthermore, the LA-CB methods generate higher test efficiency while using the sequential probability ratio test classification method.

Item selection methods with exposure and time control for computerized classification test

“…Therefore, it will be encouraging to combine the current methods with other approaches to meet various constraints simultaneously. For example, one possible solution is to replace the pure Fisher information term with the $$$ SAI $$$ index in the maximum priority index method (Cheng & Chang, 2009; Li et al, 2020).…”

cat.dt: An R package for fast construction of accurate Computerized Adaptive Tests using Decision Trees

“…Every item administered to the examinee is chosen from an item bank by employing a selection criterion that considers: i) the answers given by the participant to the items previously administered; ii) the characteristics of such items, and iii) the probabilities provided by a model that relates the responses to each item with its characteristics. The most commonly used criterion is Maximum Fisher Information (MFI) (Zhou and Reckase, 2014;Li et al, 2020), which selects the item that provides the highest information for the current estimate of the ability level. However, this criterion presents several drawbacks.…”

Two new termination rules for multidimensional computerized classification testing

“…和健康与护理问卷 (Finkelman et al, 2011;Smits & Finkelman, 2013) (Huebner & Fina, 2015;Li et al, 2020;Wang et al, 2020…”