Random intercept item factor analysis.

Maydeu-Olivares, Alberto; Coffman, Donna L.

doi:10.1037/1082-989x.11.4.344

Cited by 223 publications

(252 citation statements)

References 35 publications

Supporting

Mentioning

231

Contrasting

Unclassified

Order By: Relevance

“…In terms of the method effect specifications discussed earlier, our model is most similar to Billiet and McClendon (2000) and Welkenhuysen-Gybels et al (2003), and because of the unit loadings on the inconsistency bias factor across all items, it can be considered as a random intercept factor model (Maydeu-Olivares & Coffman, 2006). However, the model is more general because acquiescence is distinguished from careless responding, confirmation bias is taken into account, and an extension to a multi-sample context is considered so that the moderating effect of different item arrangements on the three response mechanisms can be investigated.…”

Section: An Integrative Model Of Survey Response To Regular and Revermentioning

confidence: 99%

Reversed item bias: An integrative model.

Weijters

Baumgartner

Schillewaert

2013

Psychological Methods

273

204

View full text Add to dashboard Cite

In the recent methodological literature, various models have been proposed to account for the phenomenon that reversed items (defined as items for which respondents' scores have to be recoded in order to make the direction of keying consistent across all items) tend to lead to problematic responses. In this paper we propose an integrative conceptualization of three important sources of reversed item method bias (acquiescence, careless responding, and confirmation bias) and specify a multi-sample confirmatory factor analysis model with two method factors to empirically test the hypothesized mechanisms, using explicit measures of acquiescence and carelessness and experimentally manipulated versions of a questionnaire that varies three item arrangements and the keying direction of the first item measuring the focal construct. We explain the mechanisms, review prior attempts to model reversed item bias, present our new model, and apply it to responses to a four-item self-esteem scale (N = 306) and the six-item Revised Life Orientation Test (N = 595). Based on the literature review and the empirical results, we formulate recommendations on how to use reversed items in questionnaires.Key words: reverse-keyed items, method effects, response styles, survey research, structural equation modeling. 1 Reverse-keyed items are items for which respondents' scores have to be recoded (i.e., reflected about the midpoint of the rating scale) in order for all the items in a multi-item scale to have the same directional relationship with the underlying construct of interest. The use of reverse-keyed items (also called oppositely-keyed, reversed-polarity, reverse-worded, negatively worded, negatively-keyed, keyed-false, or simply reversed items) is sometimes recommended to disrupt non-substantive responding and to enable the detection and control of aberrant response behavior when it occurs (e.g., Nunnally, 1978, Chapter 15;Paulhus, 1991).However, research has shown that reversed items often lead to problems, particularly poor model fit of factor models (e.g., Marsh, 1986). In some cases, the problem is not simply that the model based on the originally hypothesized substantive factor structure is found to be inadequate, but that the lack of fit stimulates the revision of a more parsimonious conceptualization and the specification of additional substantive factors. For example, a unidimensional model in which high and low self-esteem are considered to be opposite poles of a single underlying continuum might be rejected in favor of a model in which separate, correlated factors are posited for positive and negative self-esteem corresponding to regular and reversed items (cf. Horan, DiStefano, & Motl, 2003;Motl & DiStefano, 2002).A variety of models have been proposed to take into account differences in responding to regular and reverse-keyed items and to avoid the mistaken specification of additional substantive factors, including models with method factors or correlated uniquenesses for either the regular or the reversed items ...

show abstract

Section: An Integrative Model Of Survey Response To Regular and Revermentioning

confidence: 99%

Reversed item bias: An integrative model.

Weijters

Baumgartner

Schillewaert

2013

Psychological Methods

273

204

View full text Add to dashboard Cite

show abstract

“…One way of dealing with some simpler biases is modeling them post-completion (e.g. Maydeu-Olivares & Coffman, 2006), another way is to present questionnaire items in a comparative or forced-choice fashion. Instead of evaluating each statement in relation to a rating scale, respondents have to choose between statements according to the extent these statements describe their preferences or behavior.…”

Section: Item Response Modeling Of Forced-choice Questionnairesmentioning

confidence: 99%

Item Response Modeling of Forced-Choice Questionnaires

Brown¹,

Maydeu-Olivares

2011

Educational and Psychological Measurement

243

513

View full text Add to dashboard Cite

IRT modeling of forced choice   2 Abstract Multidimensional forced-choice formats can significantly reduce the impact of numerous response biases typically associated with rating scales. However, if scored with classical methodology these questionnaires produce ipsative data, which leads to distorted scale relationships and makes comparisons between individuals problematic. This research demonstrates how Item Response Theory (IRT) modeling may be applied to overcome these problems. A multidimensional IRT model based on Thurstone's framework for comparative data is introduced, which is suitable for use with any forced-choice questionnaire composed of items fitting the dominance response model, with any number of measured traits, and any block sizes (i.e. pairs, triplets, quads etc.). Thurstonian IRT models are normal ogive models with structured factor loadings, structured uniquenesses, and structured local dependencies.These models can be straightforwardly estimated using structural equation modeling (SEM) software Mplus. A number of simulation studies are performed to investigate how latent traits are recovered under various forced-choice designs, and to provide guidelines for optimal questionnaire design. An empirical application is given to illustrate how the model may be applied in practice. It is concluded that when the recommended design guidelines are met, scores estimated from forced-choice questionnaires with the proposed methodology reproduce the latent traits well.Keywords: forced-choice format, forced-choice questionnaires, ipsative data, comparative judgment, multidimensional IRT IRT modeling of forced choiceItem response modeling of forced-choice questionnairesThe most popular way of presenting questionnaire items is through rating scales (Likert scales), where participants are asked to rate a statement using some given categories (for example, ranging from "strongly disagree" to "strongly agree", or from "never" to "always", etc.). It is well-known that such format (single-stimulus format) can lead to various response biases, for instance because participants do not interpret the rating categories in the same way (Friedman & Amoo, 1999) between statements according to the extent these statements describe their preferences or behavior. When there are 2 statements in a block, respondents are simply asked to select the statement that better describes them. For blocks of 3, 4 or more statements, respondents may be asked to rank-order the statements, or to select one statement which is "most like me" and one which is "least like me" (i.e., to provide a partial ranking).Because it is impossible to endorse every item, the forced-choice format eliminates uniform biases such as acquiescence responding (Cheung & Chan, 2002), and can increase operational validity by reducing "halo" effects (Bartram, 2007). However, there are serious problems with the way the forced-choice questionnaires have been scored traditionally.IRT modeling of forced choiceTypically, rank orders of items in a block are reversed ...

show abstract

“…Thus, the correlated factors model can be derived from a bi-factor model by constraining the general factor loadings to zero and relaxing the orthogonality constraint on the first-order factors [25]. The bi-factor model has shown superior fit to models with first-order factors only [35]; however, this was shown using item-level data rather than subscale-level data, which may be more relevant for intelligence test scores. Furthermore, the bi-factor model may be preferred when researchers hypothesize that specific factors account for unique influence of the specific domains over and above the general factor [27].…”

Section: Introductionmentioning

confidence: 99%

Are Fit Indices Biased in Favor of Bi-Factor Models in Cognitive Ability Research?: A Comparison of Fit in Correlated Factors, Higher-Order, and Bi-Factor Models via Monte Carlo Simulations

et al. 2015

View full text Add to dashboard Cite

Bi-factor confirmatory factor models have been influential in research on cognitive abilities because they often better fit the data than correlated factors and higher-order models. They also instantiate a perspective that differs from that offered by other models. Motivated by previous work that hypothesized an inherent statistical bias of fit indices favoring the bi-factor model, we compared the fit of correlated factors, higher-order, and bi-factor models via Monte Carlo methods. When data were sampled from a true bi-factor structure, each of the approximate fit indices was more likely than not to identify the bi-factor solution as the best fitting. When samples were selected from a true multiple correlated factors structure, approximate fit indices were more likely overall to identify the correlated factors solution as the best fitting. In contrast, when samples were generated from a true higher-order structure, approximate fit indices tended to identify the bi-factor solution as best fitting. There was extensive overlap of fit values across the models regardless of true structure. Although one model may fit a given dataset best relative to the other models, each of the models tended to fit the data well in absolute terms. Given this variability, models must also be judged on substantive and conceptual grounds.

show abstract

Random intercept item factor analysis.

Cited by 223 publications

References 35 publications

Reversed item bias: An integrative model.

Reversed item bias: An integrative model.

Item Response Modeling of Forced-Choice Questionnaires

Are Fit Indices Biased in Favor of Bi-Factor Models in Cognitive Ability Research?: A Comparison of Fit in Correlated Factors, Higher-Order, and Bi-Factor Models via Monte Carlo Simulations

Contact Info

Product

Resources

About