Multidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality

Abstract

A classification method is presented for adaptive classification testing with a multidimensional item response theory (IRT) model in which items are intended to measure multiple traits, that is, within-dimensionality. The reference composite is used with the sequential probability ratio test (SPRT) to make decisions and decide whether testing can be stopped before reaching the maximum test length. Item-selection methods are provided that maximize the determinant of the information matrix at the cutoff point or at the projected ability estimate. A simulation study illustrates the efficiency and effectiveness of the classification method. Simulations were run with the new item-selection methods, random item selection, and maximization of the determinant of the information matrix at the ability estimate. The study also showed that the SPRT with multidimensional IRT has the same characteristics as the SPRT with unidimensional IRT and results in more accurate classifications than the latter when used for multidimensional data.

Look‐ahead content balancing method in variable‐length computerized classification testing

“…Firstly, different stopping rules can be evaluated and optimally determined, and the LA-CB methods can be adjusted based on the preferred stopping rule (Babcock & Weiss, 2009). It is worth examining whether the LA-CB methods can be further improved with other termination criteria including the SPRT stopping rule (van Groen, Eggen, & Veldkamp, 2016), the generalized likelihood ratio (Thompson, 2011), the fixed SEM stopping rule (Choi, Grady, & Dodd, 2011), the information stopping rule (Chang & Ying, 2004), and the projection-based stopping rules (Luo, Kim, & Dickison, 2018).…”

Multidimensional Computerized Adaptive Testing for Classifying Examinees

“…Because the equations used to find the maximum likelihood (ML) estimates have no closed-form solution, an iterative search procedure, such as Newton-Raphson, is used. Weighted maximum likelihood (Tam 1992;Van Groen et al 2016;Warm 1989) estimation reduces the bias in the ML estimates. Alternatively, Bayesian ability estimation approaches are also available (Segall 1996).…”

“…in which L θ cl + δ; x jl and L θ cl − δ; x jl are calculated using Eq. 14.2 with those items included that load on dimension l. Decision rules are then applied to the likelihood ratio to decide whether to continue testing or to make a classification decision (Seitz and Frey 2013a;Van Groen et al 2014b):…”

“…Previous research has shown that the size of α, β and sometimes also δ has a limited effect on classification accuracy (Van Groen et al 2014a, c). When classifications need to be made into one of multiple levels, the decision rules are applied sequentially for each of the cutoffs until a decision can be made (Eggen and Straetmans 2000;Van Groen et al 2014b).…”

“…Unidimensional item response theory (UIRT) is used to model the student's performance, to select the items, to decide whether testing can be stopped, and to make the classification decisions. More recently, classification methods were developed for tests measuring multiple constructs or dimensions [(i.e., multidimensionality;Seitz and Frey (2013a, b), Spray et al (1997), Van Groen et al (2014b, c, 2016] using multidimensional item response theory (MIRT). Classification decisions can be made for the entire test or for one or more dimensions or subsets of items.…”