On-the-Fly Assembled Multistage Adaptive Testing

Zheng, Yi; Chang, Hua Hua

doi:10.1177/0146621614544519

Cited by 48 publications

(45 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Fourth, the large sample size requirements for the 4PNO limit usability by applied researchers and future research should examine strategies for accurately recovering ς in smaller samples. Fifth, recent CAT research considered procedures for online calibration of item parameters (e.g., see Zheng & Chang, 2014).…”

Section: Itemmentioning

confidence: 99%

Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application

Culpepper

2015

Psychometrika

114

View full text Add to dashboard Cite

There has been renewed interest in Barton and Lord's (An upper asymptote for the three-parameter logistic item response model (Tech. Rep. No. 80-20). Educational Testing Service, 1981) four-parameter item response model. This paper presents a Bayesian formulation that extends Béguin and Glas (MCMC estimation and some model fit analysis of multidimensional IRT models. Psychometrika, 66 (4):541-561, 2001) and proposes a model for the four-parameter normal ogive (4PNO) model. Monte Carlo evidence is presented concerning the accuracy of parameter recovery. The simulation results support the use of less informative uniform priors for the lower and upper asymptotes, which is an advantage to prior research. Monte Carlo results provide some support for using the deviance information criterion and [Formula: see text] index to choose among models with two, three, and four parameters. The 4PNO is applied to 7491 adolescents' responses to a bullying scale collected under the 2005-2006 Health Behavior in School-Aged Children study. The results support the value of the 4PNO to estimate lower and upper asymptotes in large-scale surveys.

show abstract

Section: Itemmentioning

confidence: 99%

Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application

Culpepper

2015

Psychometrika

114

View full text Add to dashboard Cite

show abstract

“…The study aims at designating at which stages the average discrimination values of items that are included in the modules can be high or low in a three-stage ca-MST implementation. It is seen that item discrimination has an average value of 0.75-0.85 and SD value of 0.27-0.30 in studies that have been carried out with parameters obtained from a real pool (Kim & Moses, 2014;Kim et al, 2015, Patsula, 1999Zheng & Chang, 2015). Hambleton and Xing (2004) carried out a study in which they identified item quality as poor ( ̅ =0.60), average ( ̅ =1.00) and best corresponding to average ( ̅ =1.40) according to a parameter value.…”

Section: Methods 21 Ca-mst Panel Assemblymentioning

confidence: 99%

Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing

Şahin

2020

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

Computer Adaptive Multistage Testing (ca-MST), which take the advantage of computer technology and adaptive test form, are widely used, and are now a popular issue of assessment and evaluation. This study aims at analyzing the effect of different panel designs, module lengths, and different sequence of a parameter value across stages and change in b parameter range on measurement precision in ca-MST implementations. The study has been carried out as a simulation. MSTGen simulation software tool was used for that purpose. 5000 simulees derived from normal distribution (N (0,1)) were simulated. 60 different conditions (two panel designs (1-3-3; 1-2-2), three module lengths (10-15-20), 5 different a parameter sequences ("0.8; 0.8; 0.8"-"1.4; 0.8; 0.8"-"0.8;1.4; 0.8"-"0.8; 0.8;1,4"-"1.4; 1,4; 1.4") and two b parameter difference (small; large) conditions) were taken into consideration during analysis. Correlation, RMSE and AAD values of conditions were calculated. Conditional RMSE values corresponding to each ability level are given in a graph. Dissimilar to other studies in the literature, this study examines b parameter difference condition in three-stage tests and its interaction with a parameter sequence. Study results show that measurement precision increases as the number and length of the modules increase. Errors in measurement decrease as item discrimination values increase in all stages. Including items with a high value of item discrimination in the second or last stage contributes to measurement precision. In extreme ability levels, large difficulty difference condition produces lower error values when compared to small difficulty difference condition.

show abstract

“…Although the classical MST models have been implemented in practice for a long time, Zheng and Chang () recently proposed an alternative assembly strategy for MST: OMST. Like classical MST, OMST is constructed in several stages.…”

Section: A Hybrid Design Framework: Examples and Principlesmentioning

confidence: 99%

Hybrid Computerized Adaptive Testing: From Group Sequential Design to Fully Sequential Design

Wang

Lin²,

Chang

et al. 2016

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

Computerized adaptive testing (CAT) and multistage testing (MST) have become two of the most popular modes in large-scale computer-based sequential testing. Though most designs of CAT and MST exhibit strength and weakness in recent largescale implementations, there is no simple answer to the question of which design is better because different modes may fit different practical situations. This article proposes a hybrid adaptive framework to combine both CAT and MST, inspired by an analysis of the history of CAT and MST. The proposed procedure is a design which transitions from a group sequential design to a fully sequential design. This allows for the robustness of MST in early stages, but also shares the advantages of CAT in later stages with fine tuning of the ability estimator once its neighborhood has been identified. Simulation results showed that hybrid designs following our proposed principles provided comparable or even better estimation accuracy and efficiency than standard CAT and MST designs, especially for examinees at the two ends of the ability range.

show abstract

On-the-Fly Assembled Multistage Adaptive Testing

Cited by 48 publications

References 28 publications

Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application

Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application

Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing

Hybrid Computerized Adaptive Testing: From Group Sequential Design to Fully Sequential Design

Contact Info

Product

Resources

About