The Effect of Upper and Lower Asymptotes of IRT Models on Computerized Adaptive Testing

Cheng, Ying; Liu, Cheng

doi:10.1177/0146621615585850

Cited by 7 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Increases in amounts of data, as well as improvements in estimation quality, have seen this change somewhat, though many software packages still do not offer it as an option. The 4PL model is particularly recommended in adaptive test scenarios, where high-ability students are otherwise too heavily penalized for failing to answer questions correctly due to non-ability related reasons such as poor test conditions, unfamiliarity with computers, carelessness or misreading the question (Cheng & Liu, 2015;Kalkan, 2022;Liao et al, 2012;Rulison & Loken, 2009;Yen et al, 2012).…”

Section: Item Response Theorymentioning

confidence: 99%

bigIRT - R Software for Item Response Theory Models with Large and Sparse Data

Driver,

Tomasik

2023

Preprint

View full text Add to dashboard Cite

bigIRT is an R software package for the estimation of a range of item response theory models in contexts where there are large numbers of responses, students, and items, but the number of responses for individual students may still be low, resulting in high sparsity. The models are for binary data, using uni or multivariate, scales, 1 to 4 parameter logistic models, with covariate moderators of ability and item parameters. Estimation is maximum a posteriori, with optional empirical Bayesian approaches.

show abstract

Section: Item Response Theorymentioning

confidence: 99%

bigIRT - R Software for Item Response Theory Models with Large and Sparse Data

Driver,

Tomasik

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Magis (2013) demonstrated the computation of the item information function and the derivation of the θ value that maximizes the item information function in the 4PLM. Many other studies indicated the accuracy of ability estimates in the scoring procedure operating with the 4PLM when the slipping effects are present (e.g., Cheng & Liu, 2015; Liao et al., 2012; Rulison & Loken, 2009; Waller & Feuerstahler, 2017; Yen et al., 2012; see Table A in online supplemental materials for more information on these studies). However, the majority of studies on the 4PLM were conducted using a simulation design, specifically in computerized adaptive testing (CAT).…”

Section: Item Types Description Number Of Core Items Across 4 Forms N...mentioning

confidence: 99%

Modeling Slipping Effects in a Large‐Scale Assessment with Innovative Item Formats

Çuhadar

Binici

2022

Educational Measurement

View full text Add to dashboard Cite

This study employs the 4‐parameter logistic item response theory model to account for the unexpected incorrect responses or slipping effects observed in a large‐scale Algebra 1 End‐of‐Course assessment, including several innovative item formats. It investigates whether modeling the misfit at the upper asymptote has any practical impact on the parameter estimates. With a simulation study, it also investigates the amount of bias in the parameter estimates when the slipping effects are ignored. Findings from the empirical data indicate that the impact of ignoring slipping effects is negligible when the abilities are evaluated within the context of classification of students into performance levels; however, it is present toward the extreme ends of ability continuum within the context of individual abilities. Findings from the simulations reveal that when the proportion of items with the slipping effects is small (20%), ignoring misfit does not have practical importance; however, when the proportion of items with the slipping effects is moderate to large (50%–80%), the abilities are generally underestimated at both ends of ability scale. When an upper asymptote parameter was used for modeling the slipping effects, the items became easier and more discriminative in general than the model ignoring the slipping effects.

show abstract

“…Additionally, with the rise of computerized adaptive testing (CAT), it has been established that the 4PLM contributes to reducing bias in the early stages of CAT (Carlson, 2000;Merritt, 2003;Yen, Ho, Liao, Chen & Kuo, 2012;Chang & Ying, 2008;Cheng & Liu, 2015).…”

Section: Chapter 5: Parsimonious Item Response Theory Modeling With T...mentioning

confidence: 99%

Parsimonious item response theory modeling with different link functions

Shim¹

View full text Add to dashboard Cite

[EMBARGOED UNTIL 6/1/2023] Traditional item response theory (IRT) models assume a symmetric error distribution and rely on symmetric (logit or probit) link functions to model the response probabilities. However, this assumption does not always hold in item response data. To explore the possible benefits of alternate link functions, I developed and investigated a set of one-parameter IRT models for unidimensional tests with dichotomous items by specifying the complementary log-log (CLL), negative log-log model (NLL), and cauchit links. In a series of simulations studies, I demonstrated that these parametrically parsimonious alternate-link IRT models are comparable to traditional IRT models with regard to (1) data distribution shape, (b) inflection point shift, (c) structural similarities, and (d) response behaviors. Importantly, these four properties provide a rationale for how certain parametrically simple alternate-link models can outperform more parametrically complex traditional IRT models. Specifically, I demonstrate that the CLL model accounts for the effect of guessing because it has an item characteristic curve with a higher inflection point than that of the traditional Rasch or two-parameter logistic (2PL) models. Similarly, the NLL model addresses the effects of slipping via an inflection point that is lower than in the Rasch or 2PL models. Importantly, unlike traditional IRT models that focus on guessing and slipping as response behaviors associated with the extreme levels of the latent trait, I show that the CLL and NLL models assume such behaviors affect responses across the entire range of the trait. I also present the cauchit model, which treats both guessing and slipping effects as outliers. The simulation results reveal that these one-parameter alternate-IRT models are robust to small sample sizes (e.g., N = 100), and facilitate item-weighted scoring. I then provide further evidence for these claims by applying the CLL, NLL, and cauchit models to empirical data. Finally, I propose some extensions of alternate-link IRT modeling beyond the particular (unidimensional dichotomous) context that formed the basis of my simulations and empirical analysis. I conclude by discussing the implications of this work for applied measurement and psychometric research.

show abstract

The Effect of Upper and Lower Asymptotes of IRT Models on Computerized Adaptive Testing

Cited by 7 publications

References 15 publications

bigIRT - R Software for Item Response Theory Models with Large and Sparse Data

bigIRT - R Software for Item Response Theory Models with Large and Sparse Data

Modeling Slipping Effects in a Large‐Scale Assessment with Innovative Item Formats

Parsimonious item response theory modeling with different link functions

Contact Info

Product

Resources

About