Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-Response: Examples from Multiple Surveys

Abstract

Non-response weighting is a commonly used method to adjust for bias due to unit nonresponse in surveys. Theory and simulations show that, to reduce bias effectively without increasing variance, a covariate that is used for non-response weighting adjustment needs to be highly associated with both the response indicator and the survey outcome variable. In practice, these requirements pose a challenge that is often overlooked, because those covariates are often not observed or may not exist. Surveys have recently begun to collect supplementary data, such as interviewer observations and other proxy measures of key survey outcome variables. To the extent that these auxiliary variables are highly correlated with the actual outcomes, these variables are promising candidates for non-response adjustment. In the present study, we examine traditional covariates and new auxiliary variables for the National Survey of Family Growth, the Medical Expenditure Panel Survey, the American National Election Survey, the European Social 389 390 K r e u t e r e t a l . 1 7 3 ( 2 0 1 0 ) Surveys and the University of Michigan Transportation Research Institute survey. We provide empirical estimates of the association between proxy measures and response to the survey request as well as the actual survey outcome variables. We also compare unweighted and weighted estimates under various non-response models. Our results from multiple surveys with multiple recruitment protocols from multiple organizations on multiple topics show the difficulty of finding suitable covariates for non-response adjustment and the need to improve the quality of auxiliary data. i n J o u r n a l o f t h e ro y a l S t a t i S t i c a l S o c i e t y a

Unit Nonresponse and Weighting Adjustments: A Critical Review

“…When the information available for the sample does not predict response well, researchers have resorted to creating paradata from the survey itself (Beaumont 2005;Bates et al 2008). The use of paradata is a rapidly developing area, but initial findings reveal that this may be a difficult task (Kreuter et al 2010). …”

Multiple Auxiliary Variables in Nonresponse Adjustment

“…• Third, any given auxiliary variable is likely to differ in the strength of its association with key survey outcome variables (Kreuter et al 2010). …”

Reweighting the General Household Survey 1979–2007

“…Individual non-response weights were obtained using the response propensity scores method: a logistic regression was fitted to the probability of response and the inverse of the fitted probabilities used as weights 17,18,19 . The predictors used in the model of response to the FI section were year, age at survey, partnership status (single, married, separated, cohabiting for the years 1986-96), number of children in the household, number of adults in the household, country of birth, age of the youngest child in the household, and age of the oldest person in the household.…”