Summed Score Likelihood–Based Indices for Testing Latent Variable Distribution Fit in Item Response Theory

Li, Zhen; Cai, Li

doi:10.1177/0013164417717024

Cited by 14 publications

(23 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, there are tests that can be used for this purpose. The limited information tests of Glas (2016) and Li and Cai (2018) test whether the observed distribution of the sum score corresponds to the distribution implied by the model. An alternative approach is to compare models, by testing the marginal item response model against an augmented one with a more flexible trait distribution.…”

Section: Tests Of Model Fitmentioning

confidence: 99%

Analyzing the Fit of IRT Models With the Hausman Test

Ranger

Much

2020

Front. Psychol.

View full text Add to dashboard Cite

In this manuscript, the applicability of the Hausman test to the evaluation of item response models is investigated. The Hausman test is a general test of model fit. The test assesses whether for a model in question the parameter estimates of two different estimators coincide. The test can be implemented for item response models by comparing the parameter estimates of the marginal maximum likelihood estimator with the corresponding parameter estimates of a limited information estimator. For a correctly specified item response model, the difference of the two estimates is normally distributed around zero. The Hausman test can be used for the evaluation of item fit and global model fit. The performance of the test is evaluated in a simulation study. The simulation study suggests that the implemented versions of the test adhere to the nominal Type-I error rate well in samples of 1000 test takers and more. The test is also capable to detect misspecified item characteristic functions, but lacks power to detect violations of the conditional independence assumption. Keywords: item response theory, 2-PL model, model fit, item fit, Hausman test ANALYZING THE FIT OF IRT MODELS WITH THE HAUSMAN TESTItem response models are measurement models that allow inferring individual traits from responses given to the items of a standardized test. Core of item response models are precise assumptions about the relation between the traits and the response in a single item and the interrelation of the responses from different items. These assumptions then serve as a mathematical basis for deducing statements about a test taker's traits from his/her responses. The correctness of such inferential statements depends crucially on the validity of the item response model. As the results of psychological assessment often have important consequences, one has to guarantee that the conclusions drawn about the test takers have a sound basis. Therefore, it is indispensable to check the adequacy of the chosen item response model and its assumptions carefully. Such a check requires a powerful test of model fit.Several tests of model fit have been proposed in the past. A short overview over of the different tests is given in the following section. In doing so, the focus is mainly on the two-parameter logistic model. Nothing will be said about tests that were proposed exclusively for the Rasch model and cannot be used in general; for such tests see Glas and Verhelst (1995), Suaréz Falcón and Glas (2003), and Maydeu-Olivares and Montaño (2013. The review does also not cover the general approaches used in non-parametric item response theory (Sijtsma, 1998) or tests within the Bayesian framework (Sinharay, 2016). Tests of differential item functioning will also not be addressed (Magis et al., 2010). Having given this overview, an alternative test of model fit is Frontiers in Psychology | www.frontiersin.org 1 February 2020 | Volume 11 | Article 149 Ranger and Much Hausman Test for IRT Models Frontiers in Psychology | www.frontiersin.org

show abstract

Section: Tests Of Model Fitmentioning

confidence: 99%

Analyzing the Fit of IRT Models With the Hausman Test

Ranger

Much

2020

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…In item response theory (IRT), it is usually assumed that the probability distribution of the latent variable, say g θ , is normal. However, researchers have argued that in some situations, the normality of g θ may be a dubious assumption (Li & Cai, 2018; Reise et al, 2018; Woods, 2006; Woods & Thissen, 2006). For example, if multiple subpopulations are grouped together, then the combined g θ may be multimodal or otherwise nonnormal (Li & Cai, 2018).…”

mentioning

confidence: 99%

Testing Latent Variable Distribution Fit in IRT Using Posterior Residuals

Monroe

2020

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

This research proposes a new statistic for testing latent variable distribution fit for unidimensional item response theory (IRT) models. If the typical assumption of normality is violated, then item parameter estimates will be biased, and dependent quantities such as IRT score estimates will be adversely affected. The proposed statistic compares the specified latent variable distribution to the sample average of latent variable posterior distributions commonly used in IRT scoring. Formally, the statistic is an instantiation of a generalized residual and is thus asymptotically distributed as standard normal. Also, the statistic naturally complements residual-based item-fit statistics, as both are conditional on the latent trait, and can be presented with graphical plots. In addition, a corresponding unconditional statistic, which controls for multiple comparisons, is proposed. The statistics are evaluated using a simulation study, and empirical analyses are provided.

show abstract

“…ability distribution (e.g., Hansen, Cai, Monroe, & Li, 2014;Li & Cai, 2012). Although the nonnormality of ability distribution is included in this study, the major focus is on the sensitivity of the M 2 to the presence of the nonzero lower asymptote parameters as the ability distribution changes from normal to skewed distributions.…”

mentioning

confidence: 99%

Detection Rates of the M₂ Test for Nonzero Lower Asymptotes Under Normal and Nonnormal Ability Distributions in the Applications of IRT

Paek

Lin

2018

Applied Psychological Measurement

View full text Add to dashboard Cite

When considering the two-parameter or the three-parameter logistic model for item responses from a multiple-choice test, one may want to assess the need for the lower asymptote parameters in the item response function and make sure the use of the three-parameter item response model. This study reports the degree of sensitivity of an overall model test M 2 to detecting the presence of nonzero asymptotes in the item response function under normal and nonnormal ability distribution conditions.

show abstract

Summed Score Likelihood–Based Indices for Testing Latent Variable Distribution Fit in Item Response Theory

Cited by 14 publications

References 43 publications

Analyzing the Fit of IRT Models With the Hausman Test

Analyzing the Fit of IRT Models With the Hausman Test

Testing Latent Variable Distribution Fit in IRT Using Posterior Residuals

Detection Rates of the M₂ Test for Nonzero Lower Asymptotes Under Normal and Nonnormal Ability Distributions in the Applications of IRT

Contact Info

Product

Resources

About

Summed Score Likelihood–Based Indices for Testing Latent Variable Distribution Fit in Item Response Theory

Cited by 14 publications

References 43 publications

Analyzing the Fit of IRT Models With the Hausman Test

Analyzing the Fit of IRT Models With the Hausman Test

Testing Latent Variable Distribution Fit in IRT Using Posterior Residuals

Detection Rates of the M2 Test for Nonzero Lower Asymptotes Under Normal and Nonnormal Ability Distributions in the Applications of IRT

Contact Info

Product

Resources

About

Detection Rates of the M₂ Test for Nonzero Lower Asymptotes Under Normal and Nonnormal Ability Distributions in the Applications of IRT