Detecting Measurement Invariance in Organizational Surveys* * The original data upon which this paper is based are available at www.hhpub.com/journals/ejpa

Abstract

Summary: The problem of measurement invariance in organizational surveys is discussed, and it is shown how mixture distribution models can be used to detect response styles in organizational surveys. The results of an analysis of a leadership performance scale with the polytomous mixed Rasch model is reported (N = 4578). The results revealed that two latent classes differing in response styles could be detected: One class (size: 71%) using the whole response scale without a strong preference for specific categories and one class (size: 29%) preferring the extreme response categories and avoiding the middle ones. Furthermore, it was shown that the two latent classes differ in demographic and other organizational variables. Finally, the implications of this study for comparing individuals across divisions and organizations as well as for future research on organizational assessment methods are discussed.

Using a Mixed IRT Model to Assess the Scale Usage in the Measurement of Job Satisfaction

“…Less pronounced strategies such as avoidance of certain categories have received less attention (see for a recent overview Viswanathan et al, 2004; Van Vaerenbergh and Thomas, 2013). Eid and Rauber (2000) report that roughly a third of employees in their sample was using only five of the six presented response categories when asked to rate satisfaction with their superior. If such a misfit between the presented and the subjectively meaningful number of response categories exists, the scale will not adequately reflect the continuous underlying trait and hence violate assumptions for a rating scale (Meiser and Machunsky, 2008).…”

“…One latent class often exhibits ERS. Another one exhibits MRS when there are few response categories (e.g., 4 options) but ordinary scale usage when the number of response categories is increasing up to 6 options (Eid and Rauber, 2000; Meiser and Machunsky, 2008). …”

“…That means that within a questionnaire, one may select extreme categories for an item set and dis-/agree with other items regardless of their content. Furthermore, particularly people with ERS are also inclined to reduce the presented response format to a few subjectively meaningful categories (Eid and Rauber, 2000; Meiser and Machunsky, 2008; Wu and Huang, 2010). …”

“…Mixed IRT models such as the mixed partial credit model (mPCM; Rost, 1997) can be applied to investigate a number of scale characteristics. The focus might lie on (a) heterogeneity of scale usage (e.g., Eid and Rauber, 2000); (b) proper usage of certain response categories (e.g., Carter et al, 2011); (c) adequately representing the continuity of a trait by the response format (e.g., Meiser and Machunsky, 2008), and on (d) stability of scale usage across items, subscales or different scales (e.g., Wetzel et al, 2013). …”

“…Further, the resulting subgroup-specific latent trait values of respondents can be estimated. In contrast to raw total scores, those are adjusted to subgroup-specific scale usage and can be used to accurately compare individuals within and across latent subgroups (Eid and Rauber, 2000). …”