You Wouldn't Like Me When I'm Angry: List Experiment Misreporting^*

Abstract

Objective. List experiment respondents may misreport the number of list items that they associate with in order to associate themselves with a socially desirable test item or to disassociate themselves from a socially undesirable test item. Tests for such misreporting were conducted. Methods. List experiments from the 1991 National Race and Politics Survey, the 2006 Cooperative Congressional Election Study, and the 2008 Cooperative Campaign Analysis Project were analyzed or reviewed. Results. Evidence suggested that some respondents deflated their report more than necessary to avoid association with a socially undesirable test item. Conclusions. List experiments may provide inaccurate estimates of the percentage of the population to which the test item applies, but the direction of bias is predictable.

No magic bullet: estimating anti-immigrant sentiment and social desirability bias with the item-count technique

“…H2), ICT proved susceptible to SDB among people keen to position themselves (in the DQ) as all-out xenophiles. This subgroup's artificially low ICT score provides irrefutable evidence of deflation effects as described by Zigerell (2011). Somewhat ironically, by seeking to be "more Catholic than the Pope" these self-proclaimed xenophiles induce an underestimation of overall AIS prevalence.…”

“…In other cases, respondents may remain suspicious of the instrument 3 and offer deflated (or inflated, in the case of desirable behaviors or attitudes) item-counts to send a clear signal of disassociation from (or association with) the sensitive item. Negative ICT estimates suggest the existence of a deflation effect, whereas estimates exceeding 1 indicate an inflation effect (Zigerell 2011); still, there is scant empirical evidence to date of such distortions.…”

Asking sensitive questions using the unmatched count technique: Applications and guidelines for conservation

“…Deciding on the number of non‐sensitive items is important; typically, applications of UCT in the literature use three to four control items. While shorter lists place less cognitive burden on respondents and provide greater statistical precision, they must be carefully designed to avoid ceiling and floor effects, in order to protect individual respondents (Zigerell, ). Longer lists naturally mitigate these risks, but at the cost of lower statistical power.…”

“…However, secrecy is removed if someone reports possessing all characteristics in the treatment list, meaning that they may under‐report their true answer to avoid admitting directly to the sensitive item (“ceiling effects”). Conversely, if someone only possesses the sensitive characteristic, they may over‐report the number of non‐sensitive characteristics they possess, to conceal their answer (“floor effects”) (Zigerell, ). Ways to minimize these issues are discussed later in this paper.…”

Cooperative Survey Research

“…This award-winning paper explores questions at the intersection of political theory and political psychology and involves a collaboration of theorists and political psychologists to test core ideas about the nature of deliberation in online environments. Zigerell (2011) uses experiments embedded in a team module to explore how emotions affect survey response.…”