ACT research report series: Unidimensional approximations for a computerized test when the item pool and latent space are multidimensional

Spray, Judith A.; Abdel-fattah, Abdel-fattah A.; Huang, Chi-Yu; Lau, C. Allen

doi:10.1037/e427552008-001

Cited by 6 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research on multidimensional classification testing is still limited to the work of (Seitz and Frey 2013a, b), Spray et al (1997), and the current authors (Van Groen et al 2014b, c, 2016. Further studies will be required before multidimensional classification testing can be applied in practice.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…Unidimensional item response theory (UIRT) is used to model the student's performance, to select the items, to decide whether testing can be stopped, and to make the classification decisions. More recently, classification methods were developed for tests measuring multiple constructs or dimensions [(i.e., multidimensionality;Seitz and Frey (2013a, b), Spray et al (1997), Van Groen et al (2014b, c, 2016] using multidimensional item response theory (MIRT). Classification decisions can be made for the entire test or for one or more dimensions or subsets of items.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Multidimensional Computerized Adaptive Testing for Classifying Examinees

Groen

Eggen

Veldkamp

2019

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

Multidimensional computerized classification testing can be used when classification decisions are required for constructs that have a multidimensional structure. Here, two methods for making those decisions are included for two types of multidimensionality. In the case of between-item multidimensionality, each item is intended to measure just one dimension. In the case of within-item multidimensionality, items are intended to measure multiple or all dimensions. Wald's (1947) sequential probability ratio test and Kingsbury and Weiss (1979) confidence interval method can be applied to multidimensional classification testing. Three methods are included for selecting the items: random item selection, maximization at the current ability estimate, and the weighting method. The last method maximizes information based on a combination of the cutoff points weighted by their distance to the ability estimate. Two examples illustrate the use of the classification and item selection methods. 14.1 Introduction Computerized classification tests, like computerized adaptive tests, adapt some test elements to the student. Test length and/or item selection is tailored during test administration. The goal of a computerized classification test is to classify the examinee into one of two levels (e.g., master/nonmaster) or into one of multiple levels (e.g., basic/proficient/advanced). Classification methods stop testing when enough confi

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

Multidimensional Computerized Adaptive Testing for Classifying Examinees

Groen

Eggen

Veldkamp

2019

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

show abstract

“…Two studies about making classification decisions using MIRT exist. These studies concern MCAT with multiple unidimensional decisions for between-dimensionality (Seitz & Frey, 2013) and the use of the sequential probability ratio test (SPRT) for within-dimensionality (Spray et al, 1997).…”

Section: Classification Methodsmentioning

confidence: 99%

“…where α and β specify the acceptable classification error rates (Spray et al, 1997). A maximum test length is set to ensure that testing stops at some point (Eggen, 1999).…”

Section: Classification Methodsmentioning

confidence: 99%

“…Seitz and Frey (2013) developed a multidimensional classification method that makes a decision for each dimension for items that are assumed to measure only one trait. Spray, Abdel-Fattah, Huang, and Lau (1997) investigated classification testing for items that are assumed to measure multiple traits and concluded that this was not feasible. A new method was developed to make decisions for items that measure multiple traits.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Multidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality

Groen

Eggen

Veldkamp

2016

Applied Psychological Measurement

View full text Add to dashboard Cite

A classification method is presented for adaptive classification testing with a multidimensional item response theory (IRT) model in which items are intended to measure multiple traits, that is, within-dimensionality. The reference composite is used with the sequential probability ratio test (SPRT) to make decisions and decide whether testing can be stopped before reaching the maximum test length. Item-selection methods are provided that maximize the determinant of the information matrix at the cutoff point or at the projected ability estimate. A simulation study illustrates the efficiency and effectiveness of the classification method. Simulations were run with the new item-selection methods, random item selection, and maximization of the determinant of the information matrix at the ability estimate. The study also showed that the SPRT with multidimensional IRT has the same characteristics as the SPRT with unidimensional IRT and results in more accurate classifications than the latter when used for multidimensional data.

show abstract

Item Selection in Computerized Classification Testing

Thompson¹

2008

Educational and Psychological Measurement

View full text Add to dashboard Cite

Several alternatives for item selection algorithms based on item response theory in computerized classification testing (CCT) have been suggested, with no conclusive evidence on the substantial superiority of a single method. It is argued that the lack of sizable effect is because some of the methods actually assess items very similarly through different calculations and will usually select the same item. Consideration of methods that assess information across a wider range is often unnecessary under realistic conditions, although it might be advantageous to utilize them only early in a test. In addition, the efficiency of item selection approaches depend on the termination criteria that are used, which is demonstrated through didactic example and Monte Carlo simulation. Item selection at the cut score, which seems conceptually appropriate for CCT, is not always the most efficient option. A broad framework for item selection in CCT is presented that incorporates these points.

show abstract

ACT research report series: Unidimensional approximations for a computerized test when the item pool and latent space are multidimensional

Cited by 6 publications

References 0 publications

Multidimensional Computerized Adaptive Testing for Classifying Examinees

Multidimensional Computerized Adaptive Testing for Classifying Examinees

Multidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality

Item Selection in Computerized Classification Testing

Contact Info

Product

Resources

About