Bayesian Estimation of a Multi-Unidimensional Graded Response IRT Model

Abstract

Unidimensional graded response models are useful when items are designed to measure a unified latent trait. They are limited in practical instances where the test structure is not readily available or items are not necessarily measuring the same underlying trait. To overcome the problem, this paper proposes a multi-unidimensional normal ogive graded response model under the Bayesian framework. The performance of the proposed model was evaluated using Monte Carlo simulations. It was further compared with conventional polytomous models under simulated and real test situations. The results suggest that the proposed multi-unidimensional model is more general and flexible, and offers a better way to represent test situations not realized in unidimensional models.

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

“…Nevertheless, inferences can be made based on studies in which the sample size requirement for a more simplistic confirmatory MIRT model was investigated. Simulation studies showed that a sample size of 500 was sufficient for a between-item three-dimensional IRT model based on the graded response model, although a sample size of 250 was acceptable in limited situations (Forero & Maydeu-Olivares, 2009; Jiang, Wang, & Weiss, 2016). Based on these findings, a reasonable assumption is that more complex MIRT models, such as the two-tier, bifactor, and between-item-dimensionality IRT models with more than three dimensions, should require a sample size larger than 250, though ideally at least 500.…”

“…When the number of clusters is not a concern, obtaining complex Bayesian MIRT models suitable for sample sizes smaller than those for the aforementioned Bayesian multilevel models should be possible, although this avenue has not yet been explored for confirmatory MIRT models intended to verify complex dimensional structures in rating data of a similar size to the MRQ data (i.e., N = 121 ). For instance, Bayesian versions of the two-tier IRT model (i.e., the two-tier orthogonal model; Fujimoto, 2018), the bifactor IRT model (Fukuhara & Kamata, 2011; Sheng, 2010), and the between-item-dimensionality IRT model (Kuo & Sheng, 2015) have been presented, but these models were not specified or demonstrated to be appropriate for sample sizes as small as those of interest here.…”

A Comparison of Estimation Methods for a Multi-unidimensional Graded Response IRT Model

“…Further study can evaluate the estimation of these procedures using items with more than three scales or with different numbers of scales. Furthermore, Kuo and Sheng ( 2015 ) suggested that parameter estimate of the HwG procedure is not sensitive to different prior distributions for α . However, the selection of prior distributions for α may affect the estimation of the other estimation procedures.…”

“…Cowels ( 1996 ) proposed a HwG procedure by using a MH step within the Gibbs sampler developed by Albert and Chib ( 1993 ) for sampling the threshold parameters to improve mixing and accelerate convergence. Kuo and Sheng ( 2015 ) extended Cowels' approach to the more general multi-unidimensional model, where a constrained multivariate normal prior was assumed for θ , θ ~ N m ( 0, P ), with P being a correlation matrix to resolve the model location and scale indetermination. Moreover, the MH algorithm has been developed and implemented in BMIRT, and the BM procedure is implemented in IRTPRO for the multi-unidimensional model.…”

Using Bayesian statistical modelling as a bridge between quantitative and qualitative analyses: illustrated via analysis of an online teaching tool

“…In addition, in this case we found that a model that only included stereotyping effects for students or threshold effects for teachers did not perform as well as a model that contained both (Riley and Low-Choy, tted). As noted by Kuo and Sheng (2015), when we wish to evaluate the contribution of teacher-specific and student-specific effects at the same time, a Bayesian approach is mandated, since it is the only option: 'In IRT, simultaneous estimation of item [student] and person [teacher] parameters calls for the need of using fully Bayesian estimation via Markov Chain Monte Carlo...'. This benefit, of being able to fully address uncertainty, is named as the fourth of six benefits of a Bayesian approach for education research (Levy, 2016).…”