Measurement and control of response styles using anchoring vignettes: A model-based approach.

Bolt, Daniel M.; Lu, Yi; Kim, Jee‐Seon

doi:10.1037/met0000016

Cited by 71 publications

(147 citation statements)

References 31 publications

Supporting

Mentioning

145

Contrasting

Order By: Relevance

“…In a cross-cultural study of response style using related methods to those studied in this paper (Bolt, Lu & Kim, 2014), it was seen that different countries can make differential use rating scales in ways that do not conform to traditionally studied response style types. Thus, there would seem to be value in additional methodological study of cross-cultural differences in response style as possible sources of the attitude-achievement paradox.…”

Section: Discussionmentioning

confidence: 99%

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

Bolt

2015

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

In this paper, we consider a two-level multidimensional item response model that examines country differences in extreme response style (ERS) as a possible cause for the achievement-attitude paradox in PISA 2006. The model is an extension of Bolt & Newton (2011) that uses response data from seven attitudinal scales to assess response style and to control for its effects in estimating correlations between attitudes and achievement. Despite detectable variability in ERS across countries and detectable biasing effects of ERS on attitudinal scores, our results suggest that the unexpected between-country correlation between attitudes and achievement is not attributable to country differences in ERS. The remaining between-country correlations between mean attitudes and mean achievement once controlling for ERS can be explained by the observation that (1) despite detectable country differences, most variability in ERS occurs within, as opposed to between, countries, and (2) ERS appears to be only weakly correlated with achievement. The methodological approach used in this paper is argued to provide an informative way of studying the effects (or lack thereof) of cross-country variability in response style.

show abstract

Section: Discussionmentioning

confidence: 99%

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

Bolt

2015

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…Different families of stochastic models have been proposed to accommodate response styles in psychological assessments with common Likert response formats. On the one hand, traditional IRT models for ordinal responses such as the partial credit or rating scale model were generalized to finite mixture distribution models (e.g., Austin, Deary, & Egan, ; Eid & Rauber, ; Meiser & Machunsky, ; Rost, ; Wetzel, Carstensen, & Böhnke, ), multidimensional IRT models (e.g., Bolt, Lu, & Kim, ; Bolt & Newton, ; Falk & Cai, ; Johnson & Bolt, ; Morren, Gelissen, & Vermunt, ; Wetzel & Carstensen, ) or random‐threshold models (e.g., Jin & Wang, ; Wang, Wilson, & Shih, ; Wang & Wu, ). These extended IRT models maintain the assumption of an ordinal trait‐based response process, according to which the observed rating responses reflect gradual degrees of agreement with the item content, but account for response styles in terms of additional person parameters or in terms of discrete or continuous distributions of random threshold parameters.…”

Section: Introductionmentioning

confidence: 99%

IRTree models with ordinal and multidimensional decision nodes for response styles and trait‐based rating responses

Meiser

Plieninger

Henninger

2019

Brit J Math & Statis

View full text Add to dashboard Cite

IRTree models decompose observed rating responses into sequences of theory-based decision nodes, and they provide a flexible framework for analysing trait-related judgements and response styles. However, most previous applications of IRTree models have been limited to binary decision nodes that reflect qualitatively distinct and unidimensional judgement processes. The present research extends the family of IRTree models for the analysis of response styles to ordinal judgement processes for polytomous decisions and to multidimensional parametrizations of decision nodes. The integration of ordinal judgement processes overcomes the limitation to binary nodes, and it allows researchers to test whether decisions reflect qualitatively distinct response processes or gradual steps on a joint latent continuum. The extension to multidimensional node models enables researchers to specify multiple judgement processes that simultaneously affect the decision between competing response options. Empirical applications highlight the roles of extreme and midpoint response style in rating judgements and show that judgement processes are moderated by different response formats. Model applications with multidimensional decision nodes reveal that decisions among rating categories are jointly informed by trait-related processes and response styles.

show abstract

“…Thus, differences in how individuals respond can be viewed as differences in how the rating scale is being used. Anchoring vignette responses can be incorporated into models designed to adjust for response style effects (e.g., Bolt, Lu & Kim, 2014). While an anchoring vignette approach is appealing, it naturally requires the administration of additional items to the scales of substantive interest.…”

Section: Measuring Response Stylesmentioning

confidence: 99%

“…While an anchoring vignette approach is appealing, it naturally requires the administration of additional items to the scales of substantive interest. They also operate from a response equivalency assumption (King, et al, 2003), namely that how a respondent uses the rating scale remains consistent when responding to the anchoring vignette items, an assumption that has been empirically demonstrated to be violated in some settings (e.g., Bolt, Lu & Kim, 2014). …”

Section: Measuring Response Stylesmentioning

confidence: 99%

Extreme Response Style and the Measurement of Intra-Individual Variability in Affect

Deng

McCarthy

Piper

et al. 2018

Multivariate Behavioral Research

Self Cite

View full text Add to dashboard Cite

Extreme response style (ERS) has the potential to bias the measurement of intra-individual variability in psychological constructs (Baird, Lucas & Donnellan, 2017). This paper explores such bias through a multilevel extension of a latent trait model for modeling response styles applied to repeated measures rating scale data. Modeling responses to multi-item scales of positive and negative affect collected from smokers at clinic visits following a smoking cessation attempt revealed considerable ERS bias in the intra-individual sum score variances. In addition, simulation studies suggest the magnitude and direction of bias due to ERS is heavily dependent on the mean affect level, supporting a model-based approach to the study and control of ERS effects. Application of the proposed model-based adjustment is found to improve intra-individual variability as a predictor of smoking cessation.

show abstract

Measurement and control of response styles using anchoring vignettes: A model-based approach.

Cited by 71 publications

References 31 publications

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

IRTree models with ordinal and multidimensional decision nodes for response styles and trait‐based rating responses

Extreme Response Style and the Measurement of Intra-Individual Variability in Affect

Contact Info

Product

Resources

About