Responsive Survey Design, Demographic Data Collection, and Models of Demographic Behavior

Abstract

To address declining response rates and rising data-collection costs, survey methodologists have devised new techniques for using process data ("paradata") to address nonresponse by altering the survey design dynamically during data collection. We investigate the substantive consequences of responsive survey design-tools that use paradata to improve the representative qualities of surveys and control costs. By improving representation of reluctant respondents, responsive design can change our understanding of the topic being studied. Using the National Survey of Family Growth Cycle 6, we illustrate how responsive survey design can shape both demographic estimates and models of demographic behaviors based on survey data. By juxtaposing measures from regular and responsive data collection phases, we document how special efforts to interview reluctant respondents may affect demographic estimates. Results demonstrate the potential of responsive survey design to change the quality of demographic research based on survey data.

Do low survey response rates bias results? Evidence from Japan

“…During the fieldwork, paradata was collected at both the screening and recruiting stages. These paradata were used to produce scores for both types of non-response predicting the likelihood of converting them into respondents, and a sample of non-responders was chosen such that it had a "high mean probability of being interviewed, but including some low probability cases" (Axinn, Link, andGroves 2011: 1131). During the response phase, the following revisions were made in the fieldwork protocols: only the most productive interviewers were used; the use of proxy informants for screening questions was increased; a small incentive for completing the screening was introduced; and incentives for the actual interviews were doubled.…”

“…One lesson from the Axinn et al (2011) study is that analysts using data sets that employ a responsive design need to carefully check whether including cases collected during the response phase alters the substantive results. If the answer is "yes", then the analyst has to decide which set of results are more likely to be correct.…”

Inquérito domiciliar de distúrbios fonoaudiológicos autodeclarados: desenho e protocolo de pesquisa

“…The methodology used to conduct surveys has been studied in detail and adaptations of the methods used for selecting sample census in Brazil were incorporated (2,10,12) . The study design included data from regional surveys, methodological data of national and international surveys related to human communication and disorders (3)(4)(5)(6)(7)(8) , and specific studies related to some of the issues present in this project (1,2,(9)(10)(11)(12)(13)(14)(15) . A stratified probabilistic sampling of multiple stages (10,13) initially determined from the analysis of the age distribution in the city of Porto Alegre (southern Brazil) and representation in city neighbourhoods according to the 2000 census was performed.…”

“…Another goal was to reduce the most of refusal rates (1,12,14) , which could compromise the validity of the study findings.…”

Modeling Nonresponse in Establishment Surveys: Using an Ensemble Tree Model to Create Nonresponse Propensity Scores and Detect Potential Bias in an Agricultural Survey

“…In order to do this, it is particularly important to identify and target the low propensity groups whose nonresponse is most likely to induce nonresponse bias and to increase their response rates. Successful examples of such approaches are the National Survey of Family Growth (Axinn et al 2011), and several CATI surveys done by Statistics Canada (Mohl and Laflamme 2007;Laflamme and Karaganis 2010).…”