Inconsistent Regression and Nonresponse Bias: Exploring Their Relationship as a Function of Response Imbalance

Abstract

One objective of adaptive data collection is to secure a better balanced survey response. Methods exist for this purpose, including balancing with respect to selected auxiliary variables. Such variables are also used at the estimation stage for (calibrated) nonresponse weighting adjustment.Earlier research has shown that the use of auxiliary information at the estimation stage can reduce bias, perhaps considerably, but without eliminating it. The question is: would it have contributed further to bias reduction if, prior to estimation, that information had also been used in data collection, to secure a more balanced set of respondents? If the answer is yes, there is clear incentive, from the point of view of better accuracy in the estimates, to practice adaptive survey design, otherwise perhaps not.A key question is how the regression relationship between the survey variable and the auxiliary vector presents itself in the sample as opposed to the response. Strength in the relationship is helpful but is not the only consideration. The dilemma with nonresponse is one of inconsistent regression: a regression model appropriate for the sample often fails for the responding subset, because nonresponse is selective, non-random.In this article, we examine how nonresponse bias in survey estimates depends on regression inconsistency, both seen as functions of response imbalance. As a measure of bias we use the deviation of the calibration adjusted estimator from the unbiased estimate under full response. We study how the deviation and the regression inconsistency depend on the imbalance. We observe in empirical work that both can be reduced, to a degree, by efforts to reduce imbalance by an adaptive data collection.

Some competing nonresponse adjustment estimators

“…A regression model in s does not hold in r due to selective response mechanism, the unequal response probabilities. Bias of the calibration estimator is largely dened by the dierence in slopes b r − b s , [9]. Bias of the f-estimator is respectively dened by the dierence b sr − b s .…”

“…Nevertheless, since balancing means conditions on x-variables, the resultingȳ unw may still be biased for y s . Särndal and Lundquist [9] have conrmed that deviation ofȳ cal fromȳ s decreases, but not to zero, if balance of the response set increases.…”

Working with Response Probabilities

“…This is the situation in which the cause of missing data is completely independent of all variables measured in the survey. For more information on MCAR and other missing data mechanisms, see Little and Rubin (2002). Indeed, in the case of MCAR, self-selection does not lead to an unrepresentative sample because all elements have the same selection probability.…”

Responsive and Adaptive Design for Survey Optimization

“…In the 2017 special issue, Brick and Tourangeau (2017) present models for survey nonresponse and investigate just how effective responsive designs might be at attenuating the bias associated with those nonresponse mechanisms. Särndal and Lundquist (2017) investigate whether actively controlling the "balance" of the observed sample during the RAD data collection should be preferred to standard methods in which post-survey calibration weighting adjustments are used. Closely related to the topic of weighting calibration using sample frame and administrative data is the option to use large scale administrative data sources as a substitute for direct survey or census enumeration.…”