Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates? An Experimental Examination

Abstract

Survey research has long grappled with the concept of survey mode preference: the idea that a respondent may prefer to participate in one survey mode over another. This article experimentally examines the effect of mode preference on response, contact, and cooperation rates; mode choice; and data collection efficiency. Respondents to a 2008 telephone survey (n = 1,811; AAPOR RR3 = 38 percent) were asked their mode preference for future survey participation. These respondents were subsequently followed up in 2009 with two independent survey requests. The first follow-up survey request was another telephone survey (n = 548; AAPOR RR2 = 55.5 percent). In the second follow-up survey (n = 565; AAPOR RR2 = 46.0 percent), respondents were randomly assigned to one of four mode treatments: Web only, mail only, Web followed by mail, and mail followed by Web. We find that mode preference predicts participation in Web and phone modes, cooperation in phone mode (where contact and cooperation can be disentangled), and the selection of a mode when given the option of two modes. We find weak and mixed evidence about the relationship between mode preference and reduction of field effort. We discuss the important implications these findings have for mixed mode surveys.

Identifying predictors of survey mode preference

“…This paper starts with the observation that many surveyors believe that one possible way to increase response rates is to cater to sample members' mode preferences, and some research has suggested that this strategy can be beneficial in certain circumstances (Olson et al, 2012). However, aside from conventional wisdom, survey researchers have very little information to use in trying to determine what a sample member's mode preference might be, especially the first time a sample member is surveyed.…”

“…Giving respondents their preferred mode is thought to generate goodwill and positive attitudes toward the survey Dillman and Tarnai, 1988;Groves and Kahn, 1979;Schaefer and Dillman, 1998;Shih and Fan, 2007). One recent study lent some support to this belief, finding that respondents who prefer the internet or the telephone mode are more likely to respond to internet and phone surveys than those who do not prefer these modes (Olson et al, 2012). However, most survey researchers do not have a measure of sample members' mode preferences prior to conducting their surveys, making assignment of a 'preferred mode' difficult.…”

Estimating Components of Mean Squared Error to Evaluate the Benefits of Mixing Data Collection Modes

“…Mixed mode surveys typically involve combining modes in one of two different ways depending on the priorities of a given survey design. In "concurrent" mixed mode designs, sample members are either offered a choice between different ways of completing the survey, or particular population subgroups are targeted in a different mode to the remainder of the sample, in the hope that a preferred or more accessible mode may encourage participation (Olson et al 2012). In "sequential" mixed mode designs, the survey starts in one mode, and alternative modes are offered to nonrespondents at later stages of the fieldwork.…”

Methodological considerations for a new household panel survey

“…Furthermore, tailoring mode of data collection to an individual's stated preferred mode does not necessarily yield a higher response rate than when offering a nonpreferred mode. For example, Olson, Smyth and Wood [53] found that people who selfreport preferring a web mode in a prior wave of a survey participate at higher rates in a web survey than people who do not report preferring the web. However, all persons -including those who preferred the web mode -participated in mail and telephone surveys at higher rates than in web surveys.…”