MCMC estimation and some model-fit analysis of multidimensional IRT models

Béguin, Anton; Glas, Cornelis A.W.

doi:10.1007/bf02296195

Cited by 291 publications

(344 citation statements)

References 59 publications

Supporting

Mentioning

328

Contrasting

Unclassified

Order By: Relevance

“…These methods were specifically developed to contend with high dimensional integration and thus are well suited for the problems facing item-level factor analysis. Several researchers have already begun applying these methods in the psychological literature with some success (see Béguin & Glas, 2001;Segall, 2002;Shi & Lee, 1998 advantage of MCMC is the relative ease with which one can estimate more complex models. This is not meant to imply that MCMC is easy, but rather that there are situations where MCMC will be easier to implement than likelihood-based approaches.…”

mentioning

confidence: 99%

Item factor analysis: Current approaches and future directions.

Wirth

Edwards

2007

Psychological Methods

778

613

View full text Add to dashboard Cite

The rationale underlying factor analysis applies to continuous and categorical variables alike; however, the models and estimation methods for continuous (i.e., interval or ratio scale) data are not appropriate for item-level data that are categorical in nature. The authors provide a targeted review and synthesis of the item factor analysis (IFA) estimation literature for ordered-categorical data (e.g., Likert-type response scales) with specific attention paid to the problems of estimating models with many items and many factors. Popular IFA models and estimation methods found in the structural equation modeling and item response theory literatures are presented. Following this presentation, recent developments in the estimation of IFA parameters (e.g., Markov chain Monte Carlo) are discussed. The authors conclude with considerations for future research on IFA, simulated examples, and advice for applied researchers. Keywordsitem factor analysis; parameter estimation; categorical data; confirmatory factor analysis; item response theory Item-level data within the social and behavioral sciences are often categorical in nature. Dichotomous (e.g., disagree vs. agree) or polytomous (e.g., strongly disagree, disagree, neither, agree, and strongly agree) item response formats may fail to maintain the scale and distributional properties assumed by models such as ordinary least squares regression or common linear factor analysis. Traditional regression techniques describe the outcome variable as an optimal linear function of observed predictors. The proper implementation of these techniques requires assumptions such as independent and normally distributed residuals, a continuous conditionally normal outcome, and that the model is correctly specified (i.e., a linear relationship exists between the outcome and predictors). The common linear factor model, which describes the covariances among observed variables as a function of a smaller number of latent factors, makes many of the same assumptions. It is assumed that the unique factors (those that affect only one measured variable) are normally distributed, the outcomes are continuous and conditionally normally distributed, and a linear relationship exists between the observed and latent variables. Although this list of assumptions is not exhaustive, it does represent assumptions that are easily violated with itemlevel ordered-categorical data. Attempting to estimate model parameters, for example, NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript with dichotomous outcomes within the standard confirmatory factor model (as described by Jöreskog, 1969) results in parameter estimates that are biased and impossible to interpret accurately (Di-Stefano, 2002). However, all is not lost; just as logistic and ordinal regression techniques offer appropriate alternatives to linear regression when modeling dichotomous or polytomous (e.g., ordinal, Likert-type scales) outcomes, item factor analysis (IFA) offers an appropriate alternative to the common line...

show abstract

mentioning

confidence: 99%

Item factor analysis: Current approaches and future directions.

Wirth

Edwards

2007

Psychological Methods

778

613

View full text Add to dashboard Cite

show abstract

“…In these cases, other models can be applied to the data, such as multidimensional IRT models or latent variable mixture models. Multidimensional IRT can be modeled when multiple constructs in the data disturb the model fit (Béguin & Glas, 2001). Latent variable mixture modeling is useful in populations with heterogenous rather than homogenous samples (Sawatzky, Ratner, Kopec, Wu, & Zumbo, 2016).…”

Section: Misfit In Irt Modelsmentioning

confidence: 99%

Latent-Class-Based Item Selection for Computerized Adaptive Progress Tests

Buuren¹,

Eggen²

2017

JCAT

View full text Add to dashboard Cite

Standard computerized adaptive testing (CAT) methods require an underlying item response theory (IRT) model. An item bank can be constructed from the IRT model, and subsequent items can be selected with maximum information at the examinee's estimated ability level. IRT models, however, do not always fit test data exactly. In such situations, it is not possible to employ standard CAT methods without violating assumptions. To extend the scope of adaptive testing, this research shows how latent class analysis (LCA) can be used in item bank construction. In addition, the research investigates suitable item selection algorithms using KullbackLeibler (KL) information for item banks based on LCA. The KL information values can be used to select items and to construct an adaptive test. Simulations show that item selection based on KL information outperformed random selection of items in progress testing. The effectiveness of the selection algorithm is evaluated, and a possible scoring for the new adaptive item selection with two classes is proposed. The applicability of the methods is illustrated by constructing a computerized adaptive progress test (CAPT) on an example data set drawn from Misfit in IRT ModelsIn some situations, however, the standard IRT framework that underlies CAT does not fit the data, for example, during the calibration phase when building a CAT. Global fit of the IRT models to an item bank can be tested by looking at global fit measures such as the Q1 test, R1 test, and/or likelihood ratio tests (Andersen, 1973;Suárez-Falcón & Glas, 2003). These testing methods for global model fit can be first pointers to misfit. Concerns raised by global indications of misfit can be caused by violations of one or more assumptions of the IRT model. Item response functions can be flat instead of S-shaped, and problems with assumed local independence and unidimensionality can occur (Yang & Kao, 2014).If many items from the item bank show misfit, constructing an item bank might be impossible or only possible with heavy violations of the assumptions of standard IRT models. In these cases, other models can be applied to the data, such as multidimensional IRT models or latent variable mixture models. Multidimensional IRT can be modeled when multiple constructs in the data disturb the model fit (Béguin & Glas, 2001). Latent variable mixture modeling is useful in populations with heterogenous rather than homogenous samples (Sawatzky, Ratner, Kopec, Wu, & Zumbo, 2016). Sometimes, however, these other IRT models do not fit Research on LCA and CATThe present research appears to be the first to propose LCA in combination with CAT in an actual testing situation. Macready and Dayton (1992) investigated the use of LCA in CAT,concluding that the combination of CAT with LCA allows for conceptually simpler models than CAT with IRT. Macready and Dayton also acknowledged that LCA has fewer untestable assumptions. Their goal was to classify respondents into the appropriate latent class with CAT and to obtain an acceptable level of...

show abstract

“…The relative importance of the ability dimensions for the responses to specific items is modeled by item-specific loadings a kq and the relation between the ability dimensions in some population of respondents is modeled by the correlation between the ability dimensions. The model can be estimated and tested by various maximum likelihood and Bayesian methods (Béguin & Glas, 2001 ;Bock, Gibbons & Muraki, 1988 ;Muthén, 1984).…”

Section: Measurement Error Modelsmentioning

confidence: 99%

Item response theory in educational assessment and evaluation

Glas

2014

mee

View full text Add to dashboard Cite

Item response theory provides a useful and theoretically well-founded framework for educational measurement. It supports such activities as the construction of measurement instruments, linking and equating measurements, and evaluation of test bias and differential item functioning. It further provides underpinnings for item banking and flexible test administration designs, such as multiple matrix sampling, flexi-level testing, and computerized adaptive testing. First, a concise introduction to the principles of IRT models is given. The models discussed pertain to dichotomous items (items that are scored as either correct or incorrect) and polytomous items (items with partial credit scoring, such as most types of openended questions and performance assessments). Second, it is shown how an IRT measurement model can be enhanced with a structural model, such as, for instance, an analysis of variance model, to relate data from achievement and ability tests to students’ background variables, such as socio-economic status, intelligence or cultural capital, to school variables, and to features of the schooling system. Two applications are presented. The first one pertains to equating and linking of assessments, and the second one to a combination of an IRT measurement model and a multilevel linear model useful in school effectiveness research.La théorie de réponse à l’item (TRI) fournit un cadre utile et théoriquement bien fondé pour la mesure en éducation. Elle soutient des activités telles que la construction d’instruments de mesure, les procédures de mise en relation et de vérification d’équivalence des mesures, l’évaluation du biais d’un test et le fonctionnement différentiel d’items. Elle prévoit la base pour des banques d’items et des designs flexibles pour l’administration d’un test, comme les méthodes d’échantillonnage multicritérié, « flexi-level testing », et la méthode du test adaptatif par ordinateur. Tout d’abord, une brève introduction aux principes de modèles TRI est donnée. Les modèles discutés concernent des items dichotomiques (items qui sont corrects ou incorrects) et des items polytomiques (items à un crédit partiel, comme la plupart des questions ouvertes et questions de l’évaluation des compétences). Deuxièmement, on montre comment un modèle de mesure TRI peut être amélioré en utilisant un modèle structurel, par exemple, un modèle d’analyse de la variance, pour établir un lien entre les données provenant de tests pour mesurer le rendement et la capacité des élèves à des variables, tels leur statut socio-économique, leur niveau d’intelligence ou leur capital culturel, et à des variables caractérisant l’école et le système scolaire. Deux applications sont présentées. La première se rapporte aux procédures de type mise en parallèle (equating et linking), et la seconde à une combinaison d’un modèle de mesure TRI et d’un modèle linéaire multiniveaux utilisé dans la recherche relative à l’efficacité de l’école.A teoria de resposta ao item (TRI) fornece um quadro útil e teoricamente bem fundamentado para a...

show abstract

MCMC estimation and some model-fit analysis of multidimensional IRT models

Abstract: Bayes estimates, full-information factor analysis, Gibbs sampler, item response theory, Markov chain Monte Carlo, multidimensional item response theory, normal ogive model,

Cited by 291 publications

References 59 publications

Item factor analysis: Current approaches and future directions.

Item factor analysis: Current approaches and future directions.

Latent-Class-Based Item Selection for Computerized Adaptive Progress Tests

Item response theory in educational assessment and evaluation

Contact Info

Product

Resources

About