A General Unfolding IRT Model for Multiple Response Styles

Abstract

It is commonly known that respondents exhibit different response styles when responding to Likert-type items. For example, some respondents tend to select the extreme categories (e.g., strongly disagree and strongly agree), whereas some tend to select the middle categories (e.g., disagree, neutral, and agree). Furthermore, some respondents tend to disagree with every item (e.g., strongly disagree and disagree), whereas others tend to agree with every item (e.g., agree and strongly agree). In such cases, fitting standard unfolding item response theory (IRT) models that assume no response style will yield a poor fit and biased parameter estimates. Although there have been attempts to develop dominance IRT models to accommodate the various response styles, such models are usually restricted to a specific response style and cannot be used for unfolding data. In this study, a general unfolding IRT model is proposed that can be combined with a softmax function to accommodate various response styles via scoring functions. The parameters of the new model can be estimated using Bayesian Markov chain Monte Carlo algorithms. An empirical data set is used for demonstration purposes, followed by simulation studies to assess the parameter recovery of the new model, as well as the consequences of ignoring the impact of response styles on parameter estimators by fitting standard unfolding IRT models. The results suggest the new model to exhibit good parameter recovery and seriously biased estimates when the response styles are ignored.

From Knowledge Transmission to Knowledge Construction: A Step towards Human-Like Active Learning

“…Although it seems impossible to directly observe the level of proficiency (working knowledge), we can infer its existence through behavioral observations in a classroom. The learners are given an instrument containing several items (labeled examples) i.e., multiple-choice tests [ 27 , 28 , 50 , 51 , 52 ]. The responses to this instrument constitute the behavioral observations.…”

Fitting item response unfolding models to Likert-scale data using mirt in R

“…When using larger sample size (e.g., 2000), the bias estimates were however reduced. Notably, to help with model stability, a practical approach has been suggested to regard ρ ik = ρ k equal across items (e.g., UM3) because the common scoring rubric is used for every item [ 15 , 16 , 41 , 58 , 62 ]. Overall, the results demonstrated that GUMM2004 provided more bias when estimating the eight studied unfolding models compared to mirt with the quasi-Newton method.…”

“…The unfolding process has attracted great interests in constructs of personality, attitudes, job performance, vocational interests, leadership, emotion measurements, and so forth [ 7 – 13 ]. The unfolding model has also seen a wide variety of applications in that it has been applied to computerized adaptive testing [ 14 , 15 ], response styles [ 16 ], computerized classification testing [ 17 ], multilevel data analysis [ 18 ], multidimensional latent scaling [ 19 , 20 ], and random threshold modeling [ 15 ].…”

Estimation of Response Styles Using the Multidimensional Nominal Response Model: A Tutorial and Comparison With Sum Scores

“…Alternatively, item response models can measure response styles, and include, but are not limited to multiprocess models (e.g., Thissen-Roe and Thissen, 2013;Khorramdel and von Davier, 2014;Plieninger and Meiser, 2014;Böckenholt and Meiser, 2017), unfolding models (Liu and Wang, 2019), and the multidimensional nominal response model (MNRM; e.g., Bolt and Newton, 2011;Kieruj and Moors, 2013;Falk and Cai, 2016). Such approaches arguably rest upon testable assumptions and can handle some situations that sum scores cannot (e.g., planned missing data designs), and have numerous other advantages (e.g., conditional standard errors for score estimates).…”