Assessing Fit of Unidimensional Graded Response Models Using Bayesian Methods

Zhu, Xiaowen; Stone, Clement A.

doi:10.1111/j.1745-3984.2011.00132.x

Cited by 28 publications

(74 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The median posterior predictive p-value [c.f. Zhu and Stone, 2011] was 0.51, consistent with approximate unidimensionality. Additional details and results can be found in the online supplement .…”

Section: Resultssupporting

confidence: 55%

The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable

Sales¹,

Pane²

2019

Ann. Appl. Stat.

View full text Add to dashboard Cite

Students in Algebra I classrooms typically learn at different rates and struggle at different points in the curriculum-a common challenge for math teachers. Cognitive Tutor Algebra I (CTA1), educational computer program, addresses such student heterogeneity via what they term "mastery learning," where students progress from one section of the curriculum to the next by demonstrating appropriate "mastery" at each stage. However, when students are unable to master a section's skills even after trying many problems, they are automatically promoted to the next section anyway. Does promotion without mastery impair the program's effectiveness?At least in certain domains, CTA1 was recently shown to improve student learning on average in a randomized effectiveness study. This paper uses student log data from that study in a continuous principal stratification model to estimate the relationship between students' potential mastery and the CTA1 treatment effect. In contrast to extant principal stratification applications, a student's propensity to master worked sections here is never directly observed. Consequently we embed an item-response model, which measures students' potential mastery, within the larger principal stratification model. We find that the tutor may, in fact, be more effective for students who are more frequently promoted (despite unsuccessfully completing sections of the material). However, since these students are distinctive in their educational strength (as well as in other respects), it remains unclear whether this enhanced effectiveness can be directly attributed to aspects of the mastery learning program.

show abstract

Section: Resultssupporting

confidence: 55%

The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable

Sales¹,

Pane²

2019

Ann. Appl. Stat.

View full text Add to dashboard Cite

show abstract

“…The PPP-values are often not uniformly distributed when the fitted model is in fact correct, and there is some evidence that PPP-values under the correct model tend to be closer to 0.5 more often than would be expected under a uniform distribution (Robins et al, 2000;Sinharay et al, 2006). However, researchers such as Beguin and Glas (2001), Fox and Glas (2003), Li et al (2006), Levy and Svetina (2011), Levy et al (2009), Sinharay (2005, 2006, Sinharay et al (2006), Toribio and Albert (2011) and Zhu and Stone (2011) successfully applied the PPMC method to various types of IRT models. …”

Section: Review Of the Ppmc Methodsmentioning

confidence: 93%

Assessing person fit using l<SUP align="right">*</SUP><SUB align="right">z and the posterior predictive model checking method for dichotomous item response theory models

Sinharay¹

2015

IJQRE

View full text Add to dashboard Cite

Person-fit assessment is employed to obtain additional information regarding the answering behaviour of persons (Glas and Meijer, 2003). The lz statistic (Drasgow et al., 1985) is one of the most popular person-fit statistics. The posterior predictive model checking method (e.g., Rubin, 1984) is a popular Bayesian tool for assessing the fit of item response theory (IRT) models (e.g., Sinharay, 2005;Sinharay et al., 2006). However, the use of lz (and its modifications) combined with the PPMC method led to a conservative person-fit assessment in de la Torre and Deng (2008) and Glas and Meijer (2003). The reason of the conservativeness of this combination is provided. The use of the l * z statistic (Snijders, 2001) along with the weighted likelihood estimate (WLE; Warm, 1989) of the examinee ability and the PPMC method is suggested. The suggested approach has satisfactory power and performs as well as the best approach of de la Torre and Deng (2008). The approach is then applied to an operational dataset. Keywords: 3PL model; generalised partial credit model; IRT model fit; posterior predictive model checking; person fit.Reference to this paper should be made as follows: Sinharay, S. (2015) 'Assessing person fit using l * z and the posterior predictive model checking method for dichotomous item response theory models', Int.

show abstract

“…MCMC methods are extremely general and flexible and have proved useful in practically all aspects of Bayesian inferences, such as parameter estimation or model comparisons. Bayesian procedures have been developed for unidimensional dichotomous (Albert, 1992;Patz & Junker, 1999a,b;Sahu, 2002), multidimensional dichotomous (Beguin & Glas, 2001;Lee, 1995;Sheng & Wikle, 2007, 2009Sheng & Headrick, 2012;Yao & Boughton, 2007), and unidimensional GRM (Albert & Chib, 1993;Fox, 2010;Muraki, 1990;Zhu & Stone, 2011) models. Specifically, Sheng and Wikle (2007) described a Bayesian estimation of the multi-unidimensional IRT model for dichotomous items, where a Gibbs sampler was implemented to simultaneously estimate person/item parameters and intertrait correlations.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Estimation of a Multi-Unidimensional Graded Response IRT Model

Kuo

Sheng

2015

Behaviormetrika

View full text Add to dashboard Cite

Unidimensional graded response models are useful when items are designed to measure a unified latent trait. They are limited in practical instances where the test structure is not readily available or items are not necessarily measuring the same underlying trait. To overcome the problem, this paper proposes a multi-unidimensional normal ogive graded response model under the Bayesian framework. The performance of the proposed model was evaluated using Monte Carlo simulations. It was further compared with conventional polytomous models under simulated and real test situations. The results suggest that the proposed multi-unidimensional model is more general and flexible, and offers a better way to represent test situations not realized in unidimensional models.

show abstract

Assessing Fit of Unidimensional Graded Response Models Using Bayesian Methods

Cited by 28 publications

References 25 publications

The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable

The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable

Assessing person fit using l<SUP align="right">*</SUP><SUB align="right">z and the posterior predictive model checking method for dichotomous item response theory models

Bayesian Estimation of a Multi-Unidimensional Graded Response IRT Model

Contact Info

Product

Resources

About