Investigating Adaptive Nonresponse Follow-up Strategies for Small Businesses through Embedded Experiments

Abstract

The U.S. Census Bureau is investigating adaptive Nonresponse Follow-Up (NRFU) strategies for single unit businesses in the 2017 Economic Census. These collection protocols require a suite of viable alternative procedures that can be implemented. With business surveys, the majority of cognitive research and nonresponse follow-up procedures focus on collection methods that obtain valid response data from the larger businesses, and there is relatively little quantitative or qualitative research for small businesses. Moreover, the contact methods for small businesses are often constrained by budget limitations. Business programs at the U.S. Census Bureau rely on mailed reminder letters and supplemental promotional materials, with options for certified and bulk mailings. To explore the benefits and disadvantages of the proposed alternative nonresponse follow-up procedures for small businesses, we conducted a field experiment embedded in the 2014 Annual Survey of Manufactures, an annual program that has similar data collection procedures and sampling units as the Economic Census. This article describes the study and presents the results, then discusses how the recommended nonresponse follow-up procedures are implemented in an adaptive collection design test presently being conducted in the 2015 Annual Survey of Manufactures.

An Embedded Experiment for Targeted Non-Response Follow-Up in Establishment Surveys

“…Practically speaking, this is a very typical setting in survey methods research, with cost dominating the available strategies and quality motivating the methodologists. The collection strategy proposed builds on previous research on subsampling and collection strategies for small businesses (Kaputa and Thompson () and Thompson and Kaputa () respectively). The 2014 ASM experiment required strong support from the programme managers to implement.…”

“…Protocol 1 is the effective-but expensive-collection strategy that was recommended by the ASM programme managers, largely in response to the 2014 experiment that was described in Thompson and Kaputa (2017). All sampled units receive an initial contact letter requesting Internet response, followed by a due date reminder letter.…”

“…Consequently, the study results cannot be used to compare nonresponse collection strategy effects on relevance, accessibility and clarity, or comparability and coherence. Thompson and Kaputa (2017) examined collection strategy effects on timeliness. Biemer and Lyberg (2003) described quality in terms of total survey error, which encompasses many components, some difficult (if not impossible) to measure.…”

“…What distinguishes this field test from these other tests is the assessment of an adaptive collection design during the non-response follow-up phase, contrasting response and quality effects with those obtained under a fixed collection design. The earlier studies in this series focus on response rate effects of alternative contact strategies (Marquette et al, 2015;Thompson and Kaputa, 2017;Tuttle et al, 2018). These studies tested alternative contact strategies for small businesses; contact strategies for large businesses were out of scope for reasons outlined in Section 2.…”

Adaptive Design in Surveys and Clinical Trials: Similarities, Differences and Opportunities for Cross-fertilization

“…These studies involved changing elements of the data collection approach (subsampling of nonrespondents, altering incentive amounts, interview mode changes and contact effort changes) in response to observed sample quality and the response propensity for cases that have not yet been interviewed. In establishment surveys, these and other sorts of adaptation have been explored (Thompson and Kaputa, 2017;McCarthy et al, 2017). Särndal and Lundquist (2014) suggested that adjusting for differential recruitment needs and response behaviours during or between rounds of data collection may be more effective than not making any adaptations and instead relying on post-survey statistical adjustment to account for non-response.…”