An Adaptive Mode Adjustment for Multimode Household Surveys

Brick, J. Michael; Kennedy, Courtney; Cervantes-Flores, Ismael; Mercer, Andrew W.

doi:10.1093/jssam/smab034

Cited by 4 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The distributions of these variables based on the experimental and the control data were separately compared with the population distributions based on the 2019 American Community Survey (ACS). The unrepresentativeness is captured as deviation in the sample distribution through an imbalance (IMB) score (Brick et al 2021;Särndal and Lundquist 2019):…”

Section: Analytical Planmentioning

confidence: 99%

The Additional Effects of Adaptive Survey Design Beyond Post-Survey Adjustment: An Experimental Evaluation

Zhang

Wagner

2022

Sociological Methods & Research

View full text Add to dashboard Cite

Adaptive survey design refers to using targeted procedures to recruit different sampled cases. This technique strives to reduce bias and variance of survey estimates by trying to recruit a larger and more balanced set of respondents. However, it is not well understood how adaptive design can improve data and survey estimates beyond the well-established post-survey adjustment. This paper reports the results of an experiment that evaluated the additional effect of adaptive design to post-survey adjustments. The experiment was conducted in the Detroit Metro Area Communities Study in 2021. We evaluated the adaptive design in five outcomes: 1) response rates, 2) demographic composition of respondents, 3) bias and variance of key survey estimates, 4) changes in significant results of regression models, and 5) costs. The most significant benefit of the adaptive design was its ability to generate more efficient survey estimates with smaller variances and smaller design effects.

show abstract

Section: Analytical Planmentioning

confidence: 99%

The Additional Effects of Adaptive Survey Design Beyond Post-Survey Adjustment: An Experimental Evaluation

Zhang

Wagner

2022

Sociological Methods & Research

View full text Add to dashboard Cite

show abstract

Benefits of Adaptive Design Under Suboptimal Scenarios: A Simulation Study

Zhang

2022

Journal of Survey Statistics and Methodology

View full text Add to dashboard Cite

Theories have suggested that adaptive survey designs may reduce biases and variances of survey estimates more so than post-survey adjustment. However, previous research has not incorporated practical constraints into theorizing the utility of adaptive design. How would design factors and suboptimal auxiliary information influence the effect of adaptive design? The current simulation considers three separate components in adaptive design: (1) Modeling—how sample cases’ response propensities are predicted by a response propensity model; (2) Operationalization—how researchers decide to differentially allocate recruitment effort to sample cases, and (3) Achievement—the sizes of the changes in response propensities that are achieved by the adaptive strategies. Each component influences the effect of adaptive design on the biases and variances of survey estimates. The simulation study presented here suggests that adaptive designs based on adequate auxiliary information improve survey estimates more so than post-survey adjustment. However, adaptive designs may backfire if the strategies are not correctly designed due to a lack of critical auxiliary variables.

show abstract

Improving the Efficiency of Outbound CATI As a Nonresponse Follow-Up Mode in Address-Based Samples: A Quasi-Experimental Evaluation of a Dynamic Adaptive Design

Jackson,

Hughes,

2024

Journal of Survey Statistics and Methodology

View full text Add to dashboard Cite

This article evaluates the use of dynamic adaptive design methods to target outbound computer-assisted telephone interviewing (CATI) in the California Health Interview Survey (CHIS). CHIS is a large-scale, annual study that uses an address-based sample (ABS) with push-to-Web mailings, followed by outbound CATI follow-up for addresses with appended phone numbers. CHIS 2022 implemented a dynamic adaptive design in which predictive models were used to end dialing early for some cases. For addresses that received outbound CATI follow-up, dialing was paused after three calls. A response propensity (RP) model was applied to predict the probability that the address would respond to continued dialing, based on the outcomes of the first three calls. Low-RP addresses were permanently retired with no additional dialing, while the rest continued through six or more attempts. We use a difference-in-difference design to evaluate the effect of the adaptive design on calling effort, completion rates, and the demographic composition of respondents. We find that the adaptive design reduced the mean number of calls per sampled unit by about 14 percent (relative to a modeled no-adaptive-design counterfactual) with a minimal reduction in the completion rate and no strong evidence of changes in the prevalence of target demographics. This suggests that RP modeling can meaningfully distinguish between ABS sample units for which additional dialing is and is not productive, helping to control outbound dialing costs without compromising sample representativeness.

show abstract

An Adaptive Mode Adjustment for Multimode Household Surveys

Cited by 4 publications

References 13 publications

The Additional Effects of Adaptive Survey Design Beyond Post-Survey Adjustment: An Experimental Evaluation

The Additional Effects of Adaptive Survey Design Beyond Post-Survey Adjustment: An Experimental Evaluation

Benefits of Adaptive Design Under Suboptimal Scenarios: A Simulation Study

Improving the Efficiency of Outbound CATI As a Nonresponse Follow-Up Mode in Address-Based Samples: A Quasi-Experimental Evaluation of a Dynamic Adaptive Design

Contact Info

Product

Resources

About