A New Bayesian Person‐Fit Analysis Method Using Pivotal Discrepancy Measures

Abstract

A common method of checking person‐fit in Bayesian item response theory (IRT) is the posterior‐predictive (PP) method. In recent years, more powerful approaches have been proposed that are based on resampling methods using the popular Lz∗$L_{z}^{*}$ statistic. There has also been proposed a new Bayesian model checking method based on pivotal discrepancy measures (PDMs). A PDM T is a discrepancy measure that is a pivotal quantity with a known reference distribution. A posterior sample of T can be generated using standard Markov chain Monte Carlo output, and a p‐value is obtained from probability bounds computed on order statistics of the sample. In this paper, we propose a general procedure to apply this PDM method to person‐fit checking in IRT models. We illustrate this using the Lz$L_{z}$ and Lz∗$L_{z}^{*}$ measures. Simulation studies are done comparing these with the PP method and one of the more recent resampling methods. The results show that the PDM method is more powerful than the PP method. Under certain conditions, it is more powerful than the resampling method, while in others, it is less. The PDM method is also applied to a real data set.