Method Effects: The Problem With Negatively Versus Positively Keyed Items

Lindwall, Magnus; Barkoukis, Vassilis; Grano, Caterina; Lucidi, Fabio; Raudsepp, Lennart; Liukkonen, Jarmo; Thøgersen‐Ntoumani, Cecilie

doi:10.1080/00223891.2011.645936

Cited by 167 publications

(189 citation statements)

References 44 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…Method effects associated with NW items have received special attention in the psychometric literature (e.g., Rosenberg SelfEsteem Scale; Lindwall et al, 2012;Marsh, 1996;Tomás & Oliver, 1999). In the case of the GHQ-12, a few recent works have focused on analyzing wording effects.…”

mentioning

confidence: 99%

Wording effects and the factor structure of the 12-item General Health Questionnaire (GHQ-12).

Molina¹,

Rodrigo²,

Losilla

et al. 2014

Psychological Assessment

View full text Add to dashboard Cite

The 12-item version of the General Health Questionnaire (GHQ-12) has become a popular screening instrument with which to measure general psychological health in different settings. Previous studies into the factorial structure of the GHQ-12 have mainly supported multifactor solutions, and only a few recent works have shown that the GHQ-12 was best represented by a single substantive factor when method effects associated with negatively worded items were considered. Confirmatory factor analysis was applied to compare competing measurement models from previous research, including correlated traits, correlated methods approaches and correlated traits, correlated uniquenesses approaches, to obtain further evidence about the factorial structure of the GHQ-12. This goal was achieved with data from 3,050 participants who completed the GHQ-12 included in the Catalonian Survey of Working Conditions (Catalonian Labor Relations and Quality of Work Department, 2012). The results showed additional evidence that the GHQ-12 has a unidimensional structure after controlling for method effects associated with negatively worded items. Furthermore, we found evidence for our hypothesis about the spurious nature of the 3-factor solution in Graetz's (1991) model after comparing its fit with that found for alternative models resulting from different combinations of the negatively worded items. An implication of our results is that future research about the factor structure of the GHQ-12 should take method effects associated with negative wording into account in order to avoid reaching inaccurate conclusions about its dimensionality.

show abstract

mentioning

confidence: 99%

Wording effects and the factor structure of the 12-item General Health Questionnaire (GHQ-12).

Molina¹,

Rodrigo²,

Losilla

et al. 2014

Psychological Assessment

View full text Add to dashboard Cite

show abstract

“…contrast, other researchers consider the bi-dimensionality of a measure to be an artifact of item-keying and recommend the exclusion of reverse-keyed items (e.g., Lindwall et al, 2012;Magazine, Williams, & Williams, 1996;Schriesheim & Eisenbach, 1995;Schriesheim, Eisenbach, & Hill, 1991;van Sonderen, Sanderman, & Coyne, 2013). We believe that these decisions are being made without a clear understanding of why item-keying has the effects it does.…”

mentioning

confidence: 85%

“…Our third objective addresses the implications of the recent recommendation that researchers use measures with only regularly keyed items (e.g., Lindwall et al, 2012;Magazine et al, 1996;Schriesheim et al, 1991;Schriesheim & Eisenbach, 1995;van Sonderen et al, 2013). This recommendation is based to a large extent on the evidence of the effects of item-keying on dimensionality, but also on the desire to reduce survey length or increase the number of constructs measured in a study.…”

mentioning

confidence: 99%

Implications of Item Keying and Item Valence for the Investigation of Construct Dimensionality

Kam

Meyer

2015

Multivariate Behavioral Research

View full text Add to dashboard Cite

Factor analysis and nomological network analysis are commonly used as complementary procedures in the investigation of the dimensionality of constructs (e.g., self-esteem, job satisfaction). Although it has been demonstrated that factor analyses are often biased toward a two-dimensional solution for measures including regular- and reverse-keyed items, less attention has been paid to the implications for nomological network analyses. We propose, and demonstrate empirically in two studies, that item keying is confounded with item valence (i.e., favorability of item content), and that item valence can bias the results of both factor analysis and nomological network analysis toward a two-dimensional interpretation. We also demonstrate that the valence effect is related to, but distinguishable from, social desirability response bias. We caution that the practice of excluding reverse-keyed items to achieve unidimensionality can lead to distortion in correlations among constructs, and we offer alternative remedies to the valence problem.

show abstract

“…When method effects are not accounted for, they have been shown to affect the goodness of fit statistics, lead to biased parameter estimates (e.g., Chiu 2012Chiu , 2008Bagozzi 1993;Quilty et al 2009;Marsh 1994Marsh , 1996Distefano and Motl 2009;Marsh et al 2013;Lindwall et al 2012;Tomǭs et al 2013;Horan et al 2003;Magazine et al 1996) and influence the validity and reliability of the scale (e.g., Roszkowski and Soven 2010;Woods 2006;Raykov 2001;Brown 2015). Moreover, the method effect can lead to wrong inferences by suppressing or inflating the relationships between constructs by contributing to Type I or Type II errors if not incorporated into the measurement model (e.g., Bagozzi 1993; Magazine et al 1996).…”

Section: Methods Effects: the Negative Item Effectmentioning

confidence: 99%

TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties

Bofah

Hannula

2015

Large-scale Assess Educ

View full text Add to dashboard Cite

Relationships among motivational constructs from the 2011 Trends in International Mathematics and Science Study (TIMSS 2011) were investigated for eight-graders in all the five participating African countries, representing 38,806 (49 % girls). First, we investigated the psychometric properties (factor structure, reliabilities, method effect, and measurement invariance-country and gender) of the mathematics motivational constructs across the five educational systems. There was empirical support for the multidimensionality of the construct and the TIMSS 2011 motivational construct was largely invariant across cultures. Furthermore, a series of confirmatory factor analyses revealed that there is a need to control method effects associated with negatively worded items in the measurement model. There was support suggesting that in many cultures responses to negatively worded items are systematically different. The factor structures and reliabilities (i.e., confidence and the like mathematics scales) were affected by negatively worded items. Second, the relationships between the constructs, achievements and background variables such as parental education, gender and students' educational aspirations were investigated. We identified several significant relationships between self-belief and mathematics achievement. Differences in the latent mean achievement and the motivational construct were similar to those that have been described in the literature as "paradoxical" and "perplexing". Nations with high mathematics achievement seem to have students with more negative mathematics self-belief. Some results extend, whereas others refute the findings of previous research. For instance, the relationship between students' mathematics confidence and mathematics achievement was lower than the relationship between the value of mathematics and achievement in some countries and it was the reverse in others. However, consistent with cultural stereotypes, boys rated their mathematics competence higher than girls. The findings are discussed with reference to implications for cross-cultural research and practice.

show abstract

Method Effects: The Problem With Negatively Versus Positively Keyed Items

Cited by 167 publications

References 44 publications

Wording effects and the factor structure of the 12-item General Health Questionnaire (GHQ-12).

Wording effects and the factor structure of the 12-item General Health Questionnaire (GHQ-12).

Implications of Item Keying and Item Valence for the Investigation of Construct Dimensionality

TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties

Contact Info

Product

Resources

About