Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain

Abstract

ObjectiveWe examined the effectiveness of reverse worded items as a means of reducing or preventing response bias. We first distinguished between several types of response bias that are often confused in literature. We next developed arguments why reversing items is probably never a good way to address response bias. We proposed testing whether reverse wording affects response bias with item-level data from the Multidimensional Fatigue Inventory (MFI-20), an instrument that contains reversed worded items.MethodsWith data from 700 respondents, we compared scores on items that were similar with respect either to content or to direction of wording. Psychometric properties of sets of these items worded in the same direction were compared with sets consisting of both straightforward and reversed worded items.ResultsWe did not find evidence that ten reverse-worded items prevented response bias. Instead, the data suggest scores were contaminated by respondent inattention and confusion.ConclusionsUsing twenty items, balanced for scoring direction, to assess fatigue did not prevent respondents from inattentive or acquiescent answering. Rather, fewer mistakes are made with a 10-item instrument with items posed in the same direction. Such a format is preferable for both epidemiological and clinical studies.

Validation of the Anxiety Scale for Pregnancy in a Sample of Iranian Women

“…The lower value of alpha for baby factor may be due to the fact that it consists of both negatively and positively worded items. This result is consistent with recent evidences (37,38) against the previously recommended method of using the mixed item format (i.e. including both positively and negatively worded items) in scales (39).…”

Happiness and age in European adults: The moderating role of gross domestic product per capita.

“…Item D19 loaded poorly onto both factors, consistent with increasing evidence of generally poor performance of reverse coded items (van Sonderen, Sanderman, & Coyne, 2013), and was dropped from further analysis. Notably, CDES8 exhibited cross-loadings onto both factors (.51 and .18 respectively), but as the second cross-loading was relatively small this item was retained pending further examination.…”

Religious Orientation and Its Relationship to Suicidality: A Study in One of the Least Religious Countries

“…was discarded. As regards the reverse-coded items (also known as negatively worded items), researchers have suggested that these items may be a potential source of method bias (e.g., Podsakoff et al 2003), and the responses to these items could be contaminated by respondent's inattention and confusion (e.g., Van Sonderen et al 2013). Given the low item-total correlations of the negatively worded items (all correlation coefficients smaller than 0.4) and the presence of negative inter-item correlations between negatively worded items and positively worded items, these three negatively worded Intrinsic scale items were discarded.…”