Can Incentives Improve Survey Data Quality in Developing Countries?: Results from a Field Experiment in India

Abstract

Summary We report results of an experiment designed to assess whether the payment of contingent incentives to respondents in Karnataka, India, impacts the quality of survey data. Of 2276 households sampled at the city block level, 934 were randomly assigned to receive a small one‐time payment at the time of the survey, whereas the remaining households did not receive this incentive. We analyse the effects of incentives across a range of questions that are common in survey research in less developed countries. Our study suggests that incentives reduced unit non‐response. Conditionally on participation, we also find little impact of incentives on a broad range of sociodemographic, behavioural and attitudinal questions. In contrast, we consistently find that households that received incentives reported substantially lower consumption and income levels and fewer assets. Given random assignment and very high response rates, the most plausible interpretation of this finding is that incentivizing respondents in this setting may increase their motivation to present themselves as more needy, whether to justify the current payment or to increase the chance of receiving resources in the future. Therefore, despite early indications that contingent incentives may raise response rates, the net effect on data quality must be carefully considered.

Universal Basic Income in the Developing World

“…On strategic mis-reporting, Boozer, Goldstein and Suri (2008) for a small sample in Ghana where spouses were separately asked about consumption, show that there are some expenditures that each spouse knows nothing about, which has implications for measuring both static poverty rates as well as dynamic transitions out of poverty. Similarly, Stecklov et al (2018) use a field experiment to show that when surveys are incentivized, households report 15% less assets (with no difference in assets observable to the surveyors), which the authors interpret as households wanting to appear poorer).…”

Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys

“…A related literature uses experimental variation to test if different questionnaire designs, recall periods, or survey incentives affect reported outcomes, response rates, or data quality(Arthi et al, 2018;Beegle et al, 2012;Beaman and Dillon, 2012;Das et al, 2012;Dillon et al, 2012;Friedman et al, 2017;Gibson and Kim, 2007;Scott and Amenuvegbe, 1991;Stecklov et al, 2017).…”

A protocol for a multi-site, spatially-referenced household survey in slum settings: methods for access, sampling frame construction, sampling, and field data collection

“…Understanding the political and social contexts and obtaining buy in to the work from local stakeholders is key to the success of any research. For example, there is evidence to suggest people may misrepresent themselves if they believe doing so will result in benefits to their community [23]. Community engagement is therefore required to access the slums, ensure the reliability of the results, as well as build trust, improve communication, encourage feedback, and identify and respond to community need [24].…”