State-Dependence Effects in Surveys

Abstract

I n recent years academic research has focused on understanding and modeling the survey response process. This paper examines an understudied systematic response tendency in surveys: the extent to which observed responses are subject to state dependence, i.e., response carryover from one item to another independent of specific item content. We develop a statistical model that simultaneously accounts for state dependence, item content, and scale usage heterogeneity. The paper explores how state dependence varies by response category, item characteristics, item sequence, respondent characteristics, and whether it becomes stronger as the survey progresses. Two empirical applications provide evidence of substantial and significant state dependence. We find that the degree of state dependence depends on item characteristics and item sequence, and it varies across individuals and countries. The article demonstrates that ignoring state dependence may affect reliability and predictive validity, and it provides recommendations for survey researchers.

Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective

“…Second, the response pattern of an item measuring a specific construct frequently carries over to the subsequent item measuring (the same or) another construct due to respondents' state dependence (De Jong et al 2010). If the subsequent item is the only item measuring another construct (i.e., an SI measure), such carry-over effects might considerably affect the measure's (predictive) validity.…”

Sentence-Based Text Analysis for Customer Reviews

“…Moreover, some respondents have the expertise to provide meaningful feedback while others don't, and some provide somewhat independent evaluations about aspects of a product or service while others tend to halo their responses (Büschken et al, 2013). Respondents are also known to substitute answers to questions di↵erent than the one being posed (Gal and Rucker, 2011) and exhibit state-dependent responses where item responses carry-forward and influence later responses (de Jong et al, 2012). Conjoint analysis is similarly challenged in getting respondents to make choices that mimic marketplace sensitivities (Ding et al, 2005), i.e., to obtain coherent and valid answers to the questions posed.…”

Cultural differences in survey responding: Issues and insights in the study of response biases

“…Response sets, on the other hand, reflects a much more short‐lived response tendency. Though such transient biases have received recent attention in investigations showing that responses to one survey item may sometimes carry over to the next (e.g., de Jong, Lehmann, & Netzer, ), the focus in this section is on response styles that can be treated as more or less stable characteristics of how individuals and cultures respond to surveys (e.g., He, van de Vijver, Espinosa, & Mui, 2014b; Smith, ). Because socially desirable responding does not explicitly refer to the technical issue of how a respondent uses a response scale, it discussed in the next section.…”

“…It is standard for survey researchers interested in response biases to rely on an inferential approach to account for why respondents engage in what kinds of response styles and under what circumstances. Causal hypotheses are frequently tested by using statistically sophisticated, correlational techniques, which are being applied to already existing data (e.g., de Jong et al, ; Smith et al, ; Welkenhuysen‐Gybels et al, ). Experimental approaches are comparatively rare, presumably because the inclusion of an experimental manipulation to evoke response styles is considered too costly and undesirable in cross‐cultural surveys.…”