Using Bayesian Nonparametric Item Response Function Estimation to Check Parametric Model Fit

Abstract

Previous studies indicated that the assumption of logistic form of parametric item response functions (IRFs) is violated often enough to be worth checking. Using nonparametric item response theory (IRT) estimation methods with the posterior predictive model checking method can obtain significance probabilities of fit statistics in a Bayesian framework by accounting for the uncertainty of the parameter estimation and can indicate the location and magnitude of misfit for an item. The purpose of this study is to check the performance of the Bayesian nonparametric method to assess the IRF fit of parametric IRT models for mixed-format tests and compare it with the existing bootstrapping nonparametric method under various conditions. The simulation study results show that the Bayesian nonparametric method can detect misfit items with higher power and lower type I error rates when the sample size is large and with lower type I error rates compared with the bootstrapping method for the conditions with nonmonotonic items. In the real-data study, several dichotomous and polytomous misfit items were identified and the location and magnitude of misfit were indicated.

Bayesian Estimation of Latent Space Item Response Models with JAGS, Stan, and NIMBLE in R

“…In some cases, NIMBLE may be faster and produce better quality chains than JAGS and Stan, while in some other cases, packages such as Stan may be more efficient [30]. NIMBLE has been used for semi-parametric and non-parametric Bayesian IRT models [31,32] and other latent variable models [33]. NIMBLE has not been used for estimating LSIRM or latent space models so far.…”