Respondent Screening and Revealed Preference Axioms: Testing Quarantining Methods for Enhanced Data Quality in Web Panel Surveys

Jones, Michael; House, Lisa; Gao, Zhifeng

doi:10.1093/poq/nfv015

Cited by 105 publications

(99 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…a participant might be asked simply to check "strongly agree" to a particular item). Those who miss trap questions tend to be willing to pay more and are not as consistent in their responses, suggesting the possibility of higher error variance and even bias among people who miss trap questions (Gao, House, and Xie 2016;Jones, House, and Gao 2015;Malone and Lusk 2018a).…”

Section: A Inattentionmentioning

confidence: 99%

“…A typical trap question seeks to identify participants who briefly skim a task's instructions by providing the real subdirections hidden within the larger overall instructions (e.g., a participant might be asked simply to check “strongly agree” to a particular item). Those who miss trap questions tend to be willing to pay more and are not as consistent in their responses, suggesting the possibility of higher error variance and even bias among people who miss trap questions (Gao, House, and Xie ; Jones, House, and Gao ; Malone and Lusk ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Releasing the Trap: A Method to Reduce Inattention Bias in Survey Data With Application to U.S. Beer Taxes

Malone

Lusk

2018

Economic Inquiry

View full text Add to dashboard Cite

This study uses discrete choice experiments to explore the efficacy of prompts targeted at reducing inattention bias. Upon receiving feedback, inattentive respondents are given the opportunity to reanswer a so‐called “trap question” that checks for attentiveness. We find that individuals who miss trap questions and do not correctly revise their responses have significantly different choice patterns as compared to individuals who correctly answer the trap question. Adjusting for these inattentive responses has a substantive impact on policy impacts. Results, based on attentive participant responses, indicate that a minimum beer price would have to be substantial to substantially reduce beer demand. (JEL C83, Q18, Q51)

show abstract

Section: A Inattentionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Releasing the Trap: A Method to Reduce Inattention Bias in Survey Data With Application to U.S. Beer Taxes

Malone

Lusk

2018

Economic Inquiry

View full text Add to dashboard Cite

show abstract

“…As far as we know, Gao et al () are the first to discuss the use of VQs to detect careless respondents in online surveys and their impact on WTP estimates. Most recently Jones et al () present a paper that determines the impact of different methods of identifying the satisficing behavior on improving data. However, both Gao et al () and Jones et al () conducted their research only in the United States.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

“…Most recently Jones et al () present a paper that determines the impact of different methods of identifying the satisficing behavior on improving data. However, both Gao et al () and Jones et al () conducted their research only in the United States. The goal of this research is to use a similar method (a VQ) to compare data quality and to examine its impact on consumer WTP estimates across multiple countries.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

Online Survey Data Quality and Its Implication for Willingness‐to‐Pay: A Cross‐Country Comparison

Gao

House

Xie

2015

Canadian J Agri Economics

Self Cite

View full text Add to dashboard Cite

Using online surveys to elicit consumer preference is gaining popularity because of several advantages offered by this method. Past research mainly focuses on the comparison between online surveys and other survey modes. Few have explored methods of using online survey tools to improve data quality for consumer willingness-to-pay (WTP) estimates. This article determines the impact of using a validation question (VQ) approach that asked survey respondents to select a particular answer on improving online survey data quality across six countries. Results show that survey data quality is a common problem in online surveys across countries and the severity of this problem differs significantly. Using VQs might detect the respondents who are less careful in answering survey questions, thus providing less reliable answers. The econometric models for respondents who correctly answer VQs (pass VQs) perform significantly better than the models for respondents who incorrectly answer VQs (fail VQs). The WTP estimates for respondents who pass and fail VQs differ significantly; and in general the WTP estimates for respondents passing VQs have smaller variances than those for all respondents and for respondents failing VQs. Le recours aux enquêtes en ligne pour caractériser les préférences des consommateurs gagne en popularité en raison de ses nombreux avantages. Les recherches antérieures se sont principalement limitées a comparer les enquêtes en ligne et les autres types d'enquêtes. Peu se sont penchées sur l'utilisation d'outils susceptibles d'améliorer la qualité des données recueillies lors d'enquêtes en ligne destinéesà evaluer le consentementà payer des consommateurs. Dans le présent article, nous analysons l'impact liéà l'utilisation d'une méthode comprenant une question de validation dans laquelle on demandait aux répondants de choisir une réponse en particulier pour améliorer la qualité des données d'enquête en ligne dans six pays. Les résultats de notreétude montrent que la qualité des données d'enquête est un problème courant dans les paysà l'étude et que la gravité du problème varie considérablement d'un paysà l'autre. Le fait d'inclure des questions de validation permettrait de déceler les répondants qui sont moins consciencieux lorsqu'ils répondentà des questionsd'enquête et qui, par conséquent, fournissent des réponses plus ou moins fiables. Selon notreétude, les modèleséconométriques qui utilisent les données de répondants qui ont répondu correctement aux questions de validation donnent de meilleurs résultats que les modèles qui utilisent les données de répondants qui n'y ont pas répondu correctement; l'écart entre les estimations du consentementà payer des deux groupes de répondants varie considérablement. En règle générale, les estimations du consentementà payer des répondants qui ont répondu correctement aux questions de validation présentent de plus faibles variances que celles pour l'ensemble des répondants et pour ceux qui n'ont pas répondu correctement aux questions de validation.

show abstract

“…For example, if only 5% of the population are over-50 smokers (i.e., the study target), and 5% of the population is willing to fraudulently misrepresent itself to gain study access, roughly half of the final study sample will be frauds. In general, the rarer the incidence of a focal group in the population, the greater the proportion of study participants who are likely to be misrepresenting themselves (e.g., Jones et al 2015;Miller 2006). …”

Section: The Badmentioning

confidence: 99%

“Keep on Turkin’”?

Hulland

Miller

2018

J. of the Acad. Mark. Sci.

View full text Add to dashboard Cite

Respondent Screening and Revealed Preference Axioms: Testing Quarantining Methods for Enhanced Data Quality in Web Panel Surveys

Cited by 105 publications

References 21 publications

Releasing the Trap: A Method to Reduce Inattention Bias in Survey Data With Application to U.S. Beer Taxes

Releasing the Trap: A Method to Reduce Inattention Bias in Survey Data With Application to U.S. Beer Taxes

Online Survey Data Quality and Its Implication for Willingness‐to‐Pay: A Cross‐Country Comparison

“Keep on Turkin’”?

Contact Info

Product

Resources

About