Why Do Respondents Break Off Web Surveys and Does It Matter? Results From Four Follow-up Surveys

Steinbrecher, Markus; Roßmann, Joss; Blumenstiel, Jan Eric

doi:10.1093/ijpor/edu025

Cited by 22 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although item nonresponse may have a great impact on study validity, few studies have investigated factors related to the level of completion of internet-based epidemiologic questionnaires [12,13]. Thus, in the context of the internet-based NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) birth cohort study [14], we aimed at investigating the associations of person-level characteristics and item design factors with item nonresponse rate, as well as the associations of person-level characteristics with questionnaire breakoff.…”

Section: Introductionmentioning

confidence: 99%

Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort

Blumenberg¹,

Zugna²,

Popović³

et al. 2018

J Med Internet Res

View full text Add to dashboard Cite

BackgroundWeb-based questionnaires are increasingly used in epidemiologic studies, as traditional methods are facing a decrease in response rates and an increase in costs. However, few studies have investigated factors related to the level of completion of internet-based epidemiologic questionnaires.ObjectiveOur objective was to identify person-level characteristics and item design factors associated with breakoff (not finishing the questionnaire) and item nonresponse in a Web-based questionnaire.MethodsThis study was a cross-sectional analysis of the baseline questionnaire, applied from 2005 to 2016, of the Italian NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) birth cohort. The baseline questionnaire was administered to enrolled women, who could register at any time during pregnancy. We used logistic regression to analyze the influence of person-level factors on questionnaire breakoff, and a logistic multilevel model (first level: items of the questionnaire; second level: sections of the questionnaire; third level: study participants) to analyze the influence of person-level and item design factors on item nonresponse. Since the number of applicable items depended on the respondent’s characteristics and breakoff, we used inverse probability weighting to deal with missing by design.ResultsOf 5970 women, 519 (8.69%) did not finish the questionnaire. Older age (adjusted odds ratio 1.40, 95% CI 1.05-1.88), lower educational level (adjusted odds ratio [OR] 1.53, 95% CI 1.23-1.90), and earlier stage of pregnancy (adjusted OR 3.01, 95% CI 2.31-3.92) were positively associated with questionnaire breakoff. Of the 1,062,519 applicable items displayed for the participants, 22,831 were not responded to (overall prevalence of item nonresponse 2.15%). Item nonresponse was positively associated with older age (adjusted OR 1.25, 95% CI 1.14-1.38), being in the first trimester of pregnancy (adjusted OR 1.18, 95% CI 1.06-1.31), and lower educational level (adjusted OR 1.23, 95% CI 1.14-1.33). Dropdown menu items (adjusted OR 1.77, 95% CI 1.56-2.00) and items organized in grids (adjusted OR 1.69, 95% CI 1.49-1.91) were positively associated with item nonresponse.ConclusionsIt is important to use targeted strategies to keep participants motivated to respond. Item nonresponse in internet-based questionnaires is affected by person-level and item design factors. Some item types should be limited to reduce item nonresponse.

show abstract

Section: Introductionmentioning

confidence: 99%

Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort

Blumenberg¹,

Zugna²,

Popović³

et al. 2018

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Device effects (Display format and response entry convenience) on response quality Biemer (2010) also argued about how nonresponse error and measurement error can also be caused by different media platforms used for filling out surveys. Respondents complete questionnaires better via devices with larger screens (Steinbrecher et al, 2015). Technical problem (i.e.…”

Section: Data Quality Comparisonmentioning

confidence: 99%

Data quality comparison between computers and smartphones in different web survey modes and question formats

Zhang

Jiang

2020

INTR

View full text Add to dashboard Cite

PurposeLow response rates in web surveys and the use of different devices in entering web survey responses are the two main challenges to response quality of web surveys. The purpose of this study is to compare the effects of using interviewers to recruit participants in computer-assisted self-administered interviews (CASI) vs computer-assisted personal interviews (CAPI) and smartphones vs computers on participation rate and web survey response quality.Design/methodology/approachTwo field experiments using two similar media use studies on US college students were conducted to compare response quality in different survey modes and response devices.FindingsResponse quality of computer entry was better than smartphone entry in both studies for open-ended and closed-ended question formats. Device effect was only significant on overall completion rate when interviewers were present.Practical implicationsSurvey researchers are given guidance how to conduct online surveys using different devices and choice of question format to maximize survey response quality. The benefits and limitations of using an interviewer to recruit participants and smartphones as web survey response devices are discussed.Social implicationsIt shows how computer-assisted self-interviews and smartphones can improve response quality and participation for underprivileged groups.Originality/valueThis is the first study to compare response quality in different question formats between CASI, e-mailed delivered online surveys and CAPI. It demonstrates the importance of human factor in creating sense of obligation to improve response quality.

show abstract

“…Biemer (2010) argued that non-response error and measurement error can be caused by different devices used for survey completion. Respondents complete questionnaires better with larger screens (Steinbrecher et al , 2015). Laptop respondents are slightly more willing to type longer answers in answering open-ended questions (Lambert and Miller, 2015), but the difference is not statistically significant in another recent study (Antoun et al , 2017).…”

Section: Literature Reviewmentioning

confidence: 99%

Are computers better than smartphones for web survey responses?

Zhang

2019

OIR

View full text Add to dashboard Cite

Purpose The purpose of this paper is to examine the effect of smartphones and computers as web survey entry response devices on the quality of responses in different question formats and across different survey invitations delivery modes. The respondents’ preference of device and the response immediacy were also compared. Design/methodology/approach Two field experiments were conducted with a cluster sampling and a census of all students in a public university in the USA. Findings Device effect on response quality was only found when using computer-aided self-interviews, but not in e-mail delivered web surveys. Even though the computer was the preferred device, but the smartphone’s immediate response was significantly higher than the computer. Research limitations/implications The sample was restricted to college students who are more proficient users of smartphones and have high access to computers. But the direct comparison in the two studies using the same population increases the internal validity of the study comparing different web survey delivery modes. Practical implications Because of the minor differences in device on response quality, researchers can consider using more smartphones for field work such as computer-aided self-interviews to complement e-mail delivered surveys. Originality/value This is the first study that compares the response device effects of computer-aided self-interviews and e-mailed delivered web surveys. Because web surveys are increasingly used and various devices are being used to collect data, how respondents behave in different devices and the strengths and weaknesses of different methods of delivery survey help researchers to improve data quality and develop effective web survey delivery and participant recruitment.

show abstract

Why Do Respondents Break Off Web Surveys and Does It Matter? Results From Four Follow-up Surveys

Cited by 22 publications

References 20 publications

Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort

Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort

Data quality comparison between computers and smartphones in different web survey modes and question formats

Are computers better than smartphones for web survey responses?

Contact Info

Product

Resources

About