Data quality comparison between computers and smartphones in different web survey modes and question formats

Abstract

PurposeLow response rates in web surveys and the use of different devices in entering web survey responses are the two main challenges to response quality of web surveys. The purpose of this study is to compare the effects of using interviewers to recruit participants in computer-assisted self-administered interviews (CASI) vs computer-assisted personal interviews (CAPI) and smartphones vs computers on participation rate and web survey response quality.Design/methodology/approachTwo field experiments using two similar media use studies on US college students were conducted to compare response quality in different survey modes and response devices.FindingsResponse quality of computer entry was better than smartphone entry in both studies for open-ended and closed-ended question formats. Device effect was only significant on overall completion rate when interviewers were present.Practical implicationsSurvey researchers are given guidance how to conduct online surveys using different devices and choice of question format to maximize survey response quality. The benefits and limitations of using an interviewer to recruit participants and smartphones as web survey response devices are discussed.Social implicationsIt shows how computer-assisted self-interviews and smartphones can improve response quality and participation for underprivileged groups.Originality/valueThis is the first study to compare response quality in different question formats between CASI, e-mailed delivered online surveys and CAPI. It demonstrates the importance of human factor in creating sense of obligation to improve response quality.

Comparing Data Quality and Response Behavior Between Smartphone, Tablet, and Computer Devices in Responsive Design Online Surveys

“…Additionally, we found that the length of answers to open-ended questions was shorter on smartphones compared to computers. This finding is in line with the results of previous studies (Ha et al, 2020; Struminskaya et al, 2015). We also analyzed the level of differentiation in specific questions.…”

“…Regarding the length of responses to open-ended questions, most studies indicated shorter responses on tablets and especially on smartphones compared to computers (Ha et al, 2020;Mavletova, 2013;Struminskaya et al, 2015) or found no differences (Antoun, Katz, et al, 2017;Schlosser & Mays, 2018). Concerning distractions or multitasking behavior that can affect data quality, research found that respondents using a smartphone or a tablet seemed to show less multitasking behavior when passively measured via paradata (data on the data collection process automatically collected by the online survey software; De´cieux, 2022;Ho¨hne et al, 2020).…”

“…Thus, information from probability-based samples is needed to identify the changes in general survey participation as a result of evolving mobile optimized and mobile first designs (see e.g., Antoun, Katz, et al, 2017; Maslovskaya et al, 2019; Schlosser & Mays, 2018). Studies that implemented such a mobile-optimized design point at least to an increase in data quality with regards to specific indicators due to the better user convenience (e.g., Antoun & Cernat, 2019; Couper & Peterson, 2017; De Bruijne & Wijnant, 2014; Ha et al, 2020; Lee et al, 2018; Roßmann et al, 2018).…”

A kiosk survey of perception, attitudes and knowledge (PAK) of Australians concerning microbes, antibiotics, probiotics and hygiene

“…Another concern is that while kiosk surveys are used to evaluate business practices and customer satisfaction, they have been used in health education and public knowledge assessment projects in a limited capacity 43‐45 . It is reasonable to suggest that kiosk interaction is similar to online surveying which has in general been considered to be very useful, 46‐48 suggesting that the results here can be useful with reservations.…”