Higher-Order Item Response Models for Hierarchical Latent Traits

2016

Front. Psychol.

Self Cite

Respondents are often requested to provide a response to Likert-type or rating-scale items during the assessment of attitude, interest, and personality to measure a variety of latent traits. Extreme response style (ERS), which is defined as a consistent and systematic tendency of a person to locate on a limited number of available rating-scale options, may distort the test validity. Several latent trait models have been proposed to address ERS, but all these models have limitations. Mixture random-effect item response theory (IRT) models for ERS are developed in this study to simultaneously identify the mixtures of latent classes from different ERS levels and detect the possible differential functioning items that result from different latent mixtures. The model parameters can be recovered fairly well in a series of simulations that use Bayesian estimation with the WinBUGS program. In addition, the model parameters in the developed models can be used to identify items that are likely to elicit ERS. The results show that a long test and large sample can improve the parameter estimation process; the precision of the parameter estimates increases with the number of response options, and the model parameter estimation outperforms the person parameter estimation. Ignoring the mixtures and ERS results in substantial rank-order changes in the target latent trait and a reduced classification accuracy of the response styles. An empirical survey of emotional intelligence in college students is presented to demonstrate the applications and implications of the new models.

Section: Methodsmentioning

confidence: 83%

Mixture Random-Effect IRT Models for Controlling Extreme Response Style on Rating Scales

2016

Front. Psychol.

Self Cite

“…If a two-parameter logistic model (2PLM) or a one-parameter logistic model (1PLM) with higher order latent traits is used as the item response function at Level 1, then a two-parameter multilevel higher order IRT (2P-ML-HIRT) model and a one-parameter multilevel higher order IRT (1P-ML-HIRT) model can be formulated. Huang et al (2013) developed the polytomous HIRT model and proposed four commonly used item response models for use with polytomous items in the presence of higher order latent traits. In the framework of ML-HIRT models, for a polytomous item measuring a first-order latent trait (as in the generalized partial credit model [GPCM], Muraki, 1992, for example) at Level 1, the log odds can be defined by…”

Section: The Ml-hirt Modelmentioning

confidence: 99%

A Multilevel Higher Order Item Response Theory Model for Measuring Latent Growth in Longitudinal Data

Applied Psychological Measurement

2015

Self Cite

In educational and psychological testing, individuals are often repeatedly measured to assess the changes in their abilities over time or their latent trait growth. If a test consists of several subtests, the latent traits may have a higher order structure, and traditional item response theory (IRT) models for longitudinal data are no longer applicable. In this study, various multilevel higher order item response theory (ML-HIRT) models for simultaneously measuring growth in the second- and first-order latent traits of dichotomous and polytomous items are proposed. A series of simulations conducted using the WinBUGS software with Markov chain Monte Carlo (MCMC) methods reveal that the parameters could be recovered satisfactorily and that latent trait estimation was reliable across measurement times. The application of the ML-HIRT model to longitudinal data sets is illustrated with two empirical examples.

“…First, one can simply fit a CDM to each measurement consecutively to obtain information about latent attribute changes and estimate the CDM parameters separately for different measurement occasions. Although this approach allows the examinees’ mastery statuses to be evaluated, it ignores the association among latent attributes at different times, and the parameters are not concurrently calibrated over time, which may result in imprecise parameter estimation because cognitive skills are seldom independent in real testing situations (Tatsuoka, ) and because joint parameter calibration is more efficient than consecutive calibration in latent trait models (Huang, Wang, Chen, & Su, ).…”

mentioning

confidence: 99%

“…Fourth, to reduce the complexity of multiple correlations between attributes across times, one can assume that a higher order latent continuous trait governs the attribute mastery statuses of examinees (de la Torre & Douglas, ) and models the growth in that higher order latent trait using a multilevel approach, as in multilevel IRT models (Huang, ; Hung & Wang, ). Although the mastery statuses for the latent attributes at different times are not directly measured and are determined by the growth in the higher order latent continuous trait, the multilevel approach can be justified because an examinee with a higher level of the latent trait is more likely to exhibit a mastery status for the latent attributes (de la Torre & Douglas, )' and the relationship between the higher and lower order latent variables can be quantified by means of regression coefficients (or factor loadings) of the lower order latent traits with respect to the higher order latent trait (Huang et al., ). In addition, higher order CDMs provide not only an overall assessment (the position on the latent trait continuum) but also a fine‐grained assessment (the attribute mastery status) for examinees and thus can satisfy practical testing goals, considering that both types of information are equally important for assessment and instruction purposes.…”

mentioning

confidence: 99%

Multilevel Cognitive Diagnosis Models for Assessing Changes in Latent Attributes

J Educational Measurement

2017

Self Cite

Cognitive diagnosis models (CDMs) have been developed to evaluate the mastery status of individuals with respect to a set of defined attributes or skills that are measured through testing. When individuals are repeatedly administered a cognitive diagnosis test, a new class of multilevel CDMs is required to assess the changes in their attributes and simultaneously estimate the model parameters from the different measurements. In this study, the most general CDM of the generalized deterministic input, noisy “and” gate (G‐DINA) model was extended to a multilevel higher order CDM by embedding a multilevel structure into higher order latent traits. A series of simulations based on diverse factors was conducted to assess the quality of the parameter estimation. The results demonstrate that the model parameters can be recovered fairly well and attribute mastery can be precisely estimated if the sample size is large and the test is sufficiently long. The range of the location parameters had opposing effects on the recovery of the item and person parameters. Ignoring the multilevel structure in the data by fitting a single‐level G‐DINA model decreased the attribute classification accuracy and the precision of latent trait estimation. The number of measurement occasions had a substantial impact on latent trait estimation. Satisfactory model and person parameter recoveries could be achieved even when assumptions of the measurement invariance of the model parameters over time were violated. A longitudinal basic ability assessment is outlined to demonstrate the application of the new models.