Invitation Timing and Participation Rates in Online Panels: Findings From Two Survey Experiments

Abstract

Falling participation rates is one of the most significant challenges facing survey research today. To curb this negative trend, scholars have searched for factors that can increase and decrease citizens’ willingness to participate in surveys. In this article, we investigate the timing effects of survey invitation e-mails on participation rates in a university-based online panel with members of the Swedish public. Through two large-scale experimental studies, we examine whether the day of week ( N = 11,294) and time of day ( N = 47,279) for sending out survey invitations impact participation rates. We also ask respondents when they prefer to answer surveys. We find that the timing of survey invitations affects participation rates, however, the effects are small, short-lived, and even out within a week. We also find that the effects of timing vary by employment status and age. The results have implications for scholars and practitioners who utilize online panels for web surveys. When quick answers are important, there may be some limited gains of tailoring the timing of the survey invitation to different individuals. In surveys with more extended field periods, however, such efforts seem less warranted.

Examining the Impact of a Survey’s Email Timing on Response Latency, Mobile Response Rates, and Breakoff Rates

“…Conversely, in a survey of 24,000 scientists and engineers, Sauermann and Roach (2013) found no significant effect of email day. However, emails sent on the weekend produced a significantly slower response (consistent with Lindgren et al 2020).…”

“…Looking at individual surveys, Lewis and Hess (2017) found that Tuesday morning was the most effective time to send invitations and reminder emails for a survey of U.S. federal employees, while Faught et al (2004) found that Wednesday was best for emailing invitations to individuals sampled from U.S. manufacturing firms. Similarly, in a survey of Swedish panel members, Lindgren et al (2020) found that emailing on Wednesdays produced the highest participation rates after 24 hours and Saturdays and Sundays the lowest; however, after six days, there were no significant differences in participation rates by email send day. Conversely, in a survey of 24,000 scientists and engineers, Sauermann and Roach (2013) found no significant effect of email day.…”

“…Dillman and colleagues (2014) emphasize that all contacts should be strategically timed with the survey’s population in mind, but optimal timing varies across studies and populations. Lindgren et al (2020) suggest that email timing is important because invitations received at an inopportune time are easily forgotten or ignored. Unfortunately, relatively little recently published research examines optimal timing of web survey invitations and findings are mixed.…”

Development of a new COVID-19 panel survey: the IAB high-frequency online personal panel (HOPP)

“…In the first five waves, we invited panellists on Fridays (letters were mailed on Thursdays). From wave six on, we changed the invitation day to Monday, as we expected higher response rates by inviting people at the beginning of a week based on findings from other studies (Lindgren et al 2020 ; Blom et al 2020 ).…”

Attrition in a Longitudinal Online Randomized Controlled Trial with Prospective Homeowners

“…We also randomly assign some participants to choose the timing of their email invitations, thereby allowing participants to pick the most convenient day. To our knowledge, only Lindgren and colleagues (2018) have allowed participants to choose timing of survey invitations or reminders. They found a positive effect of sending invitations and reminders at individually preferred times, but this effect diminished over time.…”