Multiple Visits and Data Quality in Household Surveys

Abstract

In order to increase data quality some household surveys visit the respondent households several times to estimate one measure of consumption. For example, in Ghanaian Living Standards Measurement surveys, households are visited up to 10 times over a period of 1 month. I find strong evidence for conditioning effects as a result of this approach: In the Ghanaian data the estimated level of consumption is a function of the number of prior visits, with consumption being highest in the earlier survey visits. Telescoping (perceiving events as being more recent than they are) or seasonality (first‐of‐the‐month effects) cannot explain the observed pattern. To study whether earlier or later survey visits are of higher quality, I employ a strategy based on Benford's law. Results suggest that the consumption data from earlier survey visits are of higher quality than data from later visits. The findings have implications for the value of additional visits in household surveys, and also shed light on possible measurement problems in high‐frequency panels. They add to a recent literature on measurement errors in consumption surveys (Beegle et al., , Gibson et al., ), and complement findings by Zwane et al. () regarding the effect of surveys on subsequent behaviour.

Recall length and measurement error in agricultural surveys

“…The role of recall error in survey data has been explored in a range of topics, including consumption expenditure and food intake (e.g. Beegle et al, 2012b;Troubat and Grünberger, 2017;Backiny-Yetna et al, 2017;Brzozowski et al, 2017;Engle-Stone et al, 2017;D'Alessio, 2017;Schündeln, 2018;Zezza et al, 2017), household enterprises (De Mel et al, 2009;Liedholm, 1991) and income measurement (Moore et al, 2000).…”

Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys

“…Data seldom exactly follow the distribution, but statisticians routinely use the distance between the actual distribution of FSDs and the distribution under Benford's Law as a measure of data quality. See Judge and Schechter (2009) and Schündeln (2018) for examples comparing household surveys in developing countries to Benford's distribution.…”

Measuring poverty with noisy and corrected estimates of annual consumption: Evidence from Nigeria

“…Equations (4) and (6) clarify what naive extrapolation assumes but seem to require surveying households every week (month) of the year to get data to estimate In Ghana, interviewing up to 11 times a month reduced data quality with each successive interviewer visit (Schündeln, 2018), so repeat visits for a year may be even worse. Yet, instead of 66 values of r t,t ′ to form the average needed for Equation (5), a sampling approach can be used, estimating from only a few of the possible inter‐month correlations for the various i ≠ j pairs of months.…”

“…The correction method uncovers more about typical living standards because seeing the same household some months later yields new information, compared with seeing it just once or else seeing it repeatedly in short succession. Indeed, a sequence of survey visits in a short period may even harm data quality due to declining compliance (Schündeln, 2018). A potentially better use of survey resources is to have fewer total visits, and to let some months elapse between each visit.…”