The Prevalence and Implications of Slipping on Low-Stakes, Large-Scale Assessments

Abstract

In the absence of clear incentives, achievement tests may be subject to the effect of slipping where item response functions have upper asymptotes below one. Slipping reduces score precision for higher latent scores and distorts test developers' understandings of item and test information. A multidimensional four-parameter normal ogive model was developed for large-scale assessments and applied to dichotomous items of the 2011 National Assessment of Educational Progress eighth-grade mathematics and reading tests. The results suggest that the probability of slipping exceeded 5% for 47.2% and 51.1% of the dichotomous mathematics and reading items, respectively. Furthermore, allowing for slipping resulted in larger item discrimination parameters, increased information in the lower-to-middle range of the latent trait, and decreased precision for scores one standard deviation above the mean. The results provide evidence that slipping is a factor that should be considered during test development and construction to ensure adequate measurement across the latent continuum.

Performance of Person‐Fit Statistics Under Model Misspecification

“…Figures 6 and 7 show how rejection rates can be compared across item models, conditioning on T instead of θ. In practice, such plots could be constructed with more complex models for all items, with mixed format tests, or with alternative models that might fit the data better than the 3PL model (e.g., Culpepper, 2017;Lee & Bolt, 2018). Similarly, whether individual respondents are consistently flagged across alternative models can also be examined.…”

Analysing Standard Progressive Matrices (SPM-LS) with Bayesian Item Response Models

“…The most complex model I will consider in this paper is the 4-parameter logistic (4PL) model and all other simpler models result from this model by fixing some item parameters to certain values. In recent years, the 4PL model has received much attention in IRT research due to its flexibility in modeling complex binary response processes (e.g., Culpepper, 2016Culpepper, , 2017Loken & Rulison, 2010;Waller & Feuerstahler, 2017). Under this model, we express P (y ji = 1) via the equation…”

Analysing Standard Progressive Matrices (SPM-LS) with Bayesian Item Response Models

“…The most complex model I consider in this paper is the four-parameter logistic (4PL) model and all other simpler models result from this model by fixing some item parameters to certain values. In recent years, the 4PL model has received much attention in IRT research due to its flexibility in modeling complex binary response processes (e.g., Culpepper 2016Culpepper , 2017Loken and Rulison 2010;Waller and Feuerstahler 2017). Under this model, we express P(y ji = 1) via the equation…”