What’s the Benefit of a Video? The Effect of Nonmaterial Incentives on Response Rate and Bias in Web Surveys

Abstract

Traditional survey research faces declining response rates due to changing cultural habits and technological developments. Researchers have developed novel approaches to increase respondents’ likelihood of participating in web surveys. However, we lack information about whether these methods indeed increase response rates and, if so, whether they bias the resulting data. This article focuses on the use of nonmaterial incentives in the form of a video that provides the invitees with information tailored to their life situation. Analysis of our experimental data shows that instead of increasing respondents’ probability of starting the survey, the video treatments actually decrease it. We provide evidence that the lower salience of the intrinsic benefits of survey participation in the invitation email significantly contributes to this reduction. Additionally, the effect of the nonmaterial incentive differs across subgroups, affecting nonresponse biases in line with employment status, gender, and migration background. We therefore conclude that using additional information in the form of a video as a nonmaterial survey incentive is only suitable under specific conditions.

Telework and Mental Health during COVID-19

“…This might be related to the fact that women are relationally motivated to participate in social networking, such as maintaining social ties and accessing social information, while men are motivated to gain general information [ 86 ]. It is possible that women are more motivated to participate in web surveys compared to men (e.g., [ 87 ]) because they are more focused on interpersonal utility than on the personal benefits of information acquisition [ 88 ].…”

A Low-cost Method to Try to Improve Panel Survey Representation

“…Of note, Dillman (2020) has argued that knowing what to include in recruitment communications, as well as when (and with whom) to use those communications, is an understudied area in countering nonresponse. Furthermore, the existing research on the effectiveness of recruitment communications has been mostly limited to cross-sectional surveys (excluding panel maintenance communication), to recruitment text length and placement (e.g., Einarsson, Cernat, and Shlomo 2021;Martinsson, Dumitrescu, and Riedel 2017), to the application of social-psychological theory in communications (e.g., Greenberg and Dillman 2021), to the use of video (e.g., Kalleitner, Mühlböck, and Kittel 2022), and to the use of humor (e.g., Moore and Varghese 2019). The use of motivational factors in persuasive recruitment and maintenance communications (e.g., Martinsson, Dumitrescu, and Riedel 2017) has been less explored, especially when it comes to using a holistic approach, such as the one we are reporting here.…”

Conducting organizational survey and experimental research online: From convenient to ambitious in study designs, recruiting, and data quality

“…Sharing the final study results with respondents has no effect on completion rate for online studies and, oddly, seems to lower the response rate (Recklitis et al, 2009; Göritz, 2010). Thus, researchers should not try to incentivize respondents with general study results and other nonmaterial incentives (Kalleitner et al, 2020).…”