Randomized Experiments for Web‐Mail Surveys Conducted Using Address‐Based Samples of the General Population

Suzer‐Gurtekin, Z. Tuba; Elkasabi, Mahmoud; Lepkowski, James M.; Liu, Mingnan; Curtin, Richard T.

doi:10.1002/9781119083771.ch14

Cited by 5 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet mode comparisons in general population samples have found few differences in overall substantive answers between web and mail surveys, with differences primarily observed in sample composition and in Internet access and usage (see reviews in Fowler Jr. et al, 2019; Suzer-Gurtekin et al, 2019; Tourangeau et al, 2013). Previous research has found higher item nonresponse rates on mail surveys than computer web surveys (Israel & Lamm, 2012; Lesser et al, 2012; Marken et al, 2018; Messer et al, 2012; Millar & Dillman, 2012), but has not found consistent differences in item nonresponse rates or in other indicators of data quality for questionnaires completed on mobile devices versus computers after accounting for possible selection into using a particular device (e.g., Keusch & Yan, 2017; Krebs & Höhne, 2020; Lugtig & Toepoel, 2016; Lynn & Kaminska, 2012; Mavletova, 2013; Revilla & Couper, 2018; Sommer et al, 2017; Toepoel & Lugtig, 2014).…”

Section: Introductionmentioning

confidence: 99%

Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons

Olson,

Smyth,

Phillips

2023

International Journal of Market Research

View full text Add to dashboard Cite

How to display questions that are part of a battery in self-administered surveys is an important decision. Battery items may be displayed in a grid in a mail survey or computer web survey, but are often displayed as individual items on mobile devices. Although past research has compared grids to item-by-item displays in computer and mobile web surveys, almost no work has compared these displays in mail surveys. Additionally, many web survey templates use wide rectangular buttons to select response options in individual items using a mobile-optimized design, different from the standard round answer space format typically used in mail surveys. In this study, we experimentally test grid versus item-by-item displays and round radio buttons versus wide rectangular buttons for battery items in a probability-based general population mixed-mode mail + web survey of adults in Nebraska. Consistent with past research, we find that item-by-item displays reduce straightlining rates compared to grid designs. We also find that respondents are less likely to select the last two response categories in the item-by-item displays than in the grid displays. Smartphone and computer web respondents have higher item nonresponse rates than mail respondents, and web respondents have lower straightlining rates than mail respondents, accounting for respondent characteristics. Reassuringly, there is no difference in data quality outcomes across radio button versus wide button formats. These findings replicate past research that item-by-item displays reduce straightlining but may shift answer categories. These findings suggest that questionnaire designers can combine round radio button answer spaces on mail surveys with wide buttons on web surveys on battery items with little difference in data quality.

show abstract

Section: Introductionmentioning

confidence: 99%

Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons

Olson,

Smyth,

Phillips

2023

International Journal of Market Research

View full text Add to dashboard Cite

show abstract

Methods to increase response to postal and electronic questionnaires

Edwards,

Roberts,

Clarke

et al. 2023

Cochrane Database of Systematic Reviews

View full text Add to dashboard Cite

Background Self‐administered questionnaires are widely used to collect data in epidemiological research, but non‐response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research. Objectives To identify effective strategies to increase response to postal and electronic questionnaires. Search methods We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented. Selection criteria Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre‐defined criteria. Data collection and analysis We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random‐effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi 2 test and quantified the degree of inconsistency between trial results using the I 2 statistic. Main results Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I 2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I 2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I 2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follo...

show abstract

Effect of branching middle responses in dichotomous scales on web surveys

Suzer-Gurtekin

2024

International Journal of Market Research

View full text Add to dashboard Cite

Previous research shows that in telephone surveys, 11–49% of respondents select a middle alternative when it is offered on dichotomous bipolar questions, although they do not volunteer a middle option when it is not mentioned. Furthermore, offering a middle option leads to different systematic response effects, including social desirability bias and satisficing effects, which are related to respondent characteristics. Research has shown that in telephone surveys, a question form with a branching middle response has a lower response validity compared to that of a non-branching middle response. However, compared to the non-branching form, the branching form could potentially motivate respondents to spend more time and effort in their responses in the absence of an interviewer. Therefore, validity and reliability differences between branching-question form responses and non-branching-question form responses in general population web surveys have become an area of interest for research. Using a randomized experiment, this study tests the impact of a branching middle alternative in a general population survey on the validity and reliability of responses. The branching middle alternative did not change the validity and reliability of responses, and compared to the alternative, it reduced satisficing behavior based on the proxies.

show abstract

Randomized Experiments for Web‐Mail Surveys Conducted Using Address‐Based Samples of the General Population

Cited by 5 publications

References 30 publications

Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons

Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons

Methods to increase response to postal and electronic questionnaires

Effect of branching middle responses in dichotomous scales on web surveys

Contact Info

Product

Resources

About