The Bayesian Multilevel Trifactor Item Response Theory Model

Fujimoto, Ken

doi:10.1177/0013164418806694

Cited by 7 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research has demonstrated the value of using informative priors with Bayesian models when working with small samples. One simulation study showed that a sample size of 250 was sufficient to obtain accurate parameter estimates when informative prior distributions were assigned to the parameters of a Bayesian multilevel trifactor IRT model specified with eight orthogonal dimensions (Fujimoto, 2019). Another simulation study found that a sample size of 200 was enough to obtain accurate parameter estimates when informative priors were used with Bayesian multilevel structural equation models specified with four dimensions (Depaoli & Clifton, 2015; Holtmann, Koch, Lochner, & Eid, 2016).…”

Section: Sample Size Requirements For Multidimensional Item Response mentioning

confidence: 99%

“…The previously described MRQ data were analyzed to illustrate that the simulation results from the two-tier dimensional condition related to smaller sample sizes are relevant to real settings. Data were obtained from 121 students (in Grades 6-8); in-depth details of the characteristics of the sample are in Neugebauer and Fujimoto (2019). The two-tier structure hypothesized for the data was composed of primary dimensions of intrinsic and extrinsic motivation and secondary dimensions of challenge (five items), curiosity (four items), involvement (six items), competition in reading (six items), recognition in reading (four items), and grades (four items).…”

Section: Illustration With Real Datamentioning

confidence: 99%

See 1 more Smart Citation

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

Fujimoto

Neugebauer

2020

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are highly parameterized, making them only suitable for data provided by large samples. Unfortunately, many educational and psychological studies are conducted on a small scale, leaving the researchers without the necessary MIRT models to confirm the hypothesized structures in their data. To address the lack of modeling options for these researchers, we present a general Bayesian MIRT model based on adaptive informative priors. Simulations demonstrated that our MIRT model could be used to confirm a two-tier structure (with two general and six specific dimensions), a bifactor structure (with one general and six specific dimensions), and a between-item six-dimensional structure in rating scale data representing sample sizes as small as 100. Although our goal was to provide a general MIRT model suitable for smaller samples, the simulations further revealed that our model was applicable to larger samples. We also analyzed real data from 121 individuals to illustrate that the findings of our simulations are relevant to real situations.

show abstract

Section: Sample Size Requirements For Multidimensional Item Response mentioning

confidence: 99%

Section: Illustration With Real Datamentioning

confidence: 99%

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

Fujimoto

Neugebauer

2020

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the performance of the LPML has been investigated in multidimensional situations. In simulation studies in which nested-dimensionality IRT models (e.g., a bifactor IRT model) and constrained (or more parsimonious) variations of those nested-dimensionality models (e.g., a testlet IRT model) were being compared, with these studies having conditions in which each type of model was the true model, the LPML consistently identified the correct model (Fujimoto, 2018(Fujimoto, , 2019(Fujimoto, , 2020Li et al, 2006). In other words, in these studies, the LPML did not consistently favor the more complex models like the DIC.…”

Section: Log-predicted Marginal Likelihoodmentioning

confidence: 99%

The Accuracy of Bayesian Model Fit Indices in Selecting Among Multidimensional Item Response Theory Models

Fujimoto¹,

Falk²

2022

Preprint

View full text Add to dashboard Cite

Item response theory (IRT) models are often compared with respect to predictive performance to determine the dimensionality of the data gathered using educational and psychological instruments. However, such model comparisons could be biased toward nested-dimensionality IRT models (e.g., the bifactor model) when comparing those models to non-nested-dimensionality IRT models (e.g., a unidimensional or between-item-dimensionality model). The reason is that, compared to non-nested-dimensionality models, nested-dimensionality models could have a greater propensity to fit data that do not represent a specific dimensional structure. However, it is unclear as to what degree model comparison results are biased toward nested-dimensionality IRT models when the data represent specific dimensional structures and when Bayesian estimation and model comparison indices are used. We conducted a simulation study to add clarity to this issue. We examined the accuracy of four Bayesian predictive performance indices at differentiating among non-nested- and nested-dimensionality IRT models. The deviance information criterion (DIC), a commonly used index to compare Bayesian models, was extremely biased toward nested-dimensionality IRT models, favoring them even when non-nested-dimensionality models were the correct models. The Pareto-smoothed importance sampling approximation of the leave-one-out cross-validation was the least biased, with the Watanabe information criterion and the log-predicted marginal likelihood closely following. The findings demonstrate that nested-dimensionality IRT models are not automatically favored for data representing specific dimensional structures as long as an appropriate predictive performance index is used.

show abstract

“…If the primary dimension is only weakly represented in the data, then a structure other than a bifactor structure would be more appropriate for the data. The third reason for these starting values is that they have been shown to lead to stable solutions across a range of sample sizes when they were used in a different multilevel multidimensional IRT model (Fujimoto, ).…”

Section: The Bayesian Correlated Multilevel Bifactor Irt Modelmentioning

confidence: 99%

“…These models, therefore, can be viewed as special cases of the multilevel multidimensional IRT modeling framework (Muthén & Asparouhov, ; Raudenbush, Johnson, & Sampson, ). There are many variations of dual‐dependent IRT models, such as those based on multilevel extensions of the cross‐classified structure (Jiao, Kamata, & Xie, ) and the trifactor structure (Fujimoto, ), but of particular interest for this study are the variations based on the multilevel bifactor structure (De Jong, Steenkamp, & Fox, ; Jiao et al., ; Jiao & Zhang, ; Wang, Kim, Dedrick, Ferron, & Tan, ). Thus, for the remainder of this article, dual‐dependent IRT models will refer to those versions based on a multilevel bifactor structure, with the multilevel portion consisting of the responses (Level 1) nested within persons (Level 2) and the persons nested within clusters (Level 3).…”

Section: A Conceptual Overview Of Dual‐dependent Irt Models' Assumptionsmentioning

confidence: 99%

A More Flexible Bayesian Multilevel Bifactor Item Response Theory Model

Fujimoto

2019

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

Multilevel bifactor item response theory (IRT) models are commonly used to account for features of the data that are related to the sampling and measurement processes used to gather those data. These models conventionally make assumptions about the portions of the data structure that represent these features. Unfortunately, when data violate these models' assumptions but these models are used anyway, incorrect conclusions about the cluster effects could be made and potentially relevant dimensions could go undetected. To address the limitations of these conventional models, a more flexible multilevel bifactor IRT model that does not make these assumptions is presented, and this model is based on the generalized partial credit model. Details of a simulation study demonstrating this model outperforming competing models and showing the consequences of using conventional multilevel bifactor IRT models to analyze data that violate these models' assumptions are reported. Additionally, the model's usefulness is illustrated through the analysis of the Program for International Student Assessment data related to interest in science.

show abstract

The Bayesian Multilevel Trifactor Item Response Theory Model

Cited by 7 publications

References 35 publications

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples

The Accuracy of Bayesian Model Fit Indices in Selecting Among Multidimensional Item Response Theory Models

A More Flexible Bayesian Multilevel Bifactor Item Response Theory Model

Contact Info

Product

Resources

About