Releasing the Trap: A Method to Reduce Inattention Bias in Survey Data With Application to U.S. Beer Taxes

Abstract

This study uses discrete choice experiments to explore the efficacy of prompts targeted at reducing inattention bias. Upon receiving feedback, inattentive respondents are given the opportunity to reanswer a so‐called “trap question” that checks for attentiveness. We find that individuals who miss trap questions and do not correctly revise their responses have significantly different choice patterns as compared to individuals who correctly answer the trap question. Adjusting for these inattentive responses has a substantive impact on policy impacts. Results, based on attentive participant responses, indicate that a minimum beer price would have to be substantial to substantially reduce beer demand. (JEL C83, Q18, Q51)

How to run an experimental auction: a review of recent advances

“…Cason and Plott (2014) show that although many subjects did not bid their induced value, they did state their correct valuation in a second round of bidding after they were exposed to their mistake by rereading the instructions and after receiving feedback. This is consistent with the findings in Malone and Lusk (2018) where in a discrete choice experiment they provide feedback to inattentive respondents which are subsequently given the opportunity to re-answer a 'trap question' that checks for attentiveness. In Malone and Lusk (2018) individuals who do not correctly revise their responses after missing a trap question have significantly different choice patterns than individuals who correctly answer the trap question.…”

Inattention, availability bias, and attribute premium estimation for a biobased product

“…Respondents unfamiliar with a product may lose interest in completing choice tasks, which could lead to survey fatigue or incorrect responses (Malone & Lusk, 2019). The psychology literature finds that incorrect responses to obvious yes/no questions signal respondent inattention (Berinsky et al, 2014).…”

“…They found that individuals who incorrectly answered trap questions responded differently. Malone and Lusk (2019) used double trap questions to identify inattentive respondents. They concluded that individuals who answered incorrectly on both trap questions exhibited significantly different preferences.…”

Is respondents' inattention in online surveys a major issue for research?

“…Participant is instructed to ignore the response format and select a specific answer (Berinsky et al, 2014) "Please ignore the question below about how you are feeling and instead check only the 'none of the above' option as your answer"; "Please click on the word that describes how you are currently feeling" (Malone & Lusk, 2019) Post-hoc analysis…”