A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

Abstract

Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are highly parameterized, making them only suitable for data provided by large samples. Unfortunately, many educational and psychological studies are conducted on a small scale, leaving the researchers without the necessary MIRT models to confirm the hypothesized structures in their data. To address the lack of modeling options for these researchers, we present a general Bayesian MIRT model based on adaptive informative priors. Simulations demonstrated that our MIRT model could be used to confirm a two-tier structure (with two general and six specific dimensions), a bifactor structure (with one general and six specific dimensions), and a between-item six-dimensional structure in rating scale data representing sample sizes as small as 100. Although our goal was to provide a general MIRT model suitable for smaller samples, the simulations further revealed that our model was applicable to larger samples. We also analyzed real data from 121 individuals to illustrate that the findings of our simulations are relevant to real situations.

HBMIRT: A SAS Macro for Estimating Uni- and Multidimensional 1- and 2-Parameter Item Response Models in Small (and Large!) Samples (R1)

“…The estimation of such models, however, typically requires rather large sample sizes which makes it difficult for applied researchers in many fields to make use of IRT. This drawback can be overcome by Bayesian IRT approaches with hierarchical priors that allow to estimate (multidimensional) IRT models in rather small samples as recent studies have demonstrated (Fujimoto & Neugebauer, 2020;Gilholm et al, 2021;König et al, 2020;Sheng, 2013). To date, such approaches might be under-used by applied researchers for two reasons: First, researchers may not be aware of the advantages of the Bayesian IRT framework with hierarchical priors.…”

“…Often, so called uninformative (or diffuse, flat, vague) priors are specified when no information is a priori available about the parameters to be estimated, which can lead to severe bias when samples are small (Smid et al, 2020;Zitzmann, Lüdtke, et al, 2021). HBMIRT SAS MACRO 4 To mitigate this problem, the use of hierarchical prior distributions (also called adaptive informative priors) has specifically been suggested for IRT models when these models are used in small samples (Fujimoto & Neugebauer, 2020;Gilholm et al, 2021;König et al, 2020;Sheng, 2013). The idea behind this approach is that distributions for the parameters of a type of prior for model parameters (e.g., for a set of item discrimination parameters) are assumed.…”

“…This information can be used for model comparisons using a pseudo Bayes factor approach (Geisser & Eddy, 1979). It should be noted that this approach is a very reliable model selection approach for IRT models with hierarchical priors (Fujimoto & Neugebauer, 2020) and even more reliable than the deviance information criterion (DIC; Spiegelhalter et al, 2002), which is generated by default as -2 • M(log likelihood) + 2 • Var(log likelihood) from the second half of the chain 2 .…”

Using National Data to Explore Online and Offline Reading Comprehension Processes

“…The priors assigned to the parameters were those shown to be suitable for IRT models when used to analyze data based on sample sizes of at least 100—under a wide range of conditions (Fujimoto & Neugebauer, 2020). A normal distribution with a mean of 0 and standard deviation ( SD ) of 1 was assigned to the abilities, .…”

A Latent Auto-Regressive Approach for Bayesian Structural Equation Modeling of Spatially or Socially Dependent Data

“…The derivation of such models with factor score methods or maximum likelihood based approaches is not straightforward and needs special attention (see such approaches for nonlinear, non-spatial SEM; e.g., Klein & Moosbrugger, 2000;Wall & Amemiya, 2000). Bayesian methods have become a very powerful statistical approach to a wide range of complex methods, including multilevel modeling (Asparouhov & Muth en, 2016), longitudinal data analyses (Asparouhov, Hamaker, & Muth en, 2017;Schultzberg & Muth en, 2018), and latent variable modeling in general (e.g., Arminger & Muth en, 1998;Fox, 2010;Fujimoto & Neugebauer, 2020;Lee, 2007). Bayesian SEM have been introduced some time ago, and their application has recently become more feasible with new user-friendly statistical packages (e.g., Asparouhov & Muth en, 2010;Merkle & Rosseel, 2018).…”