A randomized experiment evaluating survey mode effects for video interviewing

Endres, Kyle; Hillygus, D. Sunshine; DeBell, Matthew; Iyengar, Shanto

doi:10.1017/psrm.2022.30

Cited by 7 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As with all data collection methods, a challenge with mixed-mode and innovative methods is ensuring the quality of the survey data. Mixing modes results in survey data being collected under very different conditions (e.g., interviewer presence or absence, verbal or visual presentation of question stimuli, differing question formats); thus, mode effects can affect data quality and survey estimates (Conrad et al, 2022;de Leeuw & Hox, 2015;Endres et al, 2022;Lugtig et al, 2011;West et al, 2022). Moreover, when relatively new data collection methods are used that are unfamiliar to both interviewers and respondents, such as CAVI, little is known about the problems that may occur during the interview, such as interrupted speech and frozen or distorted video (Conrad et al, 2022), and about the impact of the new interview situation and the problems encountered on response behavior and data quality.…”

Section: Challenges In Data Qualitymentioning

confidence: 99%

“…Even if the technical requirements are met, not all respondents are ready for and comfortable with a video interview. Like KtN, CAVI involves comprehensive scheduling (Endres et al, 2022; Schober et al, 2020). Respondents’ varying ability and willingness to participate and the more complex call scheduling, particularly for CAVI and KtN, underscore the importance of tailored fieldwork management, propensity modeling, and nonresponse bias analysis.…”

Section: Interviewer-observed Paradata In Mixed-mode and Innovative D...mentioning

confidence: 99%

“…In recent years, and further reinforced by the COVID-19 pandemic, there have been increasing calls in the market and social research to switch from face-to-face-only interviewing to mixed-mode designs (Luijkx et al, 2021; Wolf et al, 2021), or other innovative survey data collection methods (Conrad et al, 2022; Endres et al, 2022; Jeannis et al, 2013; Schober, 2018; West et al, 2022). While the idea of mixed-mode data collection and its benefits are not new (de Leeuw, 2005; 2018; Dillman, 2005; Scherpenzeel, 2017), they have become even more important in the post-pandemic era (Cleary et al, 2021; Kuenzi et al, 2022; Kantar Public, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Interviewer-Observed Paradata in Mixed-Mode and Innovative Data Collection

Kunz

Daikeler

Ackermann-Piek

2023

International Journal of Market Research

View full text Add to dashboard Cite

In this research note, we address the potentials of using interviewer-observed paradata, typically collected during face-to-face-only interviews, in mixed-mode and innovative data collection methods that involve an interviewer at some stage (e.g., during the initial contact or during the interview). To this end, we first provide a systematic overview of the types and purposes of the interviewer-observed paradata most commonly collected in face-to-face interviews—contact form data, interviewer observations, and interviewer evaluations—using the methodology of evidence mapping. Based on selected studies, we illustrate the main purposes of interviewer-observed paradata we identified—including fieldwork management, propensity modeling, nonresponse bias analysis, substantive analysis, and survey data quality assessment. Based on this, we discuss the possible use of interviewer-observed paradata in mixed-mode and innovative data collection methods. We conclude with thoughts on new types of interviewer-observed paradata and the potential of combining paradata from different survey modes.

show abstract

Section: Challenges In Data Qualitymentioning

confidence: 99%

Section: Interviewer-observed Paradata In Mixed-mode and Innovative D...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Interviewer-Observed Paradata in Mixed-Mode and Innovative Data Collection

Kunz

Daikeler

Ackermann-Piek

2023

International Journal of Market Research

View full text Add to dashboard Cite

show abstract

“…Research has tested mode effects extensively in the context of Europe and North America (e.g., Heerwegh 2009;Atkeson et al 2014;Aquilino 1994;Pasek 2016; Endres et al 2023) and increasingly in developing country contexts (e.g., Boas et al 2020;Greenleaf et al 2020).…”

mentioning

confidence: 99%

The Interaction of Survey Mode and Social Desirability in Measuring Behavior during Wartime

Erlich

2024

Preprint

View full text Add to dashboard Cite

Measuring individuals' behavior from survey data during an active war raises distinct challenges. To investigate survey mode and social desirability effects, we conduct a multi-mode study (opt-in web, RDD telephone) with two independently drawn samples of the Ukrainian population during Russia's full-scale invasion of Ukraine in the summer of 2022. We employ identical demographic and behavioral questions in both surveys to investigate cross-mode reliability. We also embed a pre-registered priming experiment to examine whether respondents may inflate prosocial behavior. Observationally, we find that because web studies cannot reach the oldest Ukrainians, they likely contain significant population bias with respect to war-related demographic variables. Experimentally, we find evidence of inflation in reporting volunteering activities only in the telephone survey. Our findings demonstrate that, given modern survey techniques, wartime attitudes and behavior can be reliably measured, but there is a trade-off in survey modes between representativeness and response bias. The manuscript contributes to our knowledge about measuring behavior and survey methods.

show abstract

“…The focus is on willingness to participate rather than actual participation, as part of exploring the viability of live video surveys, based on evidence from other technologies that the behavioral intention to use a technology can directly affect actual usage behavior (Davis & Wiedenbeck, 2001). The point of the study is not to generalize to a full demographic analysis of U.S. preferences at the moment of data collection, nor to test the quality of survey responses in live video versus other modes in actual interviews (e.g., Conrad et al, 2022;Endres et al, 2023)-which are currently rare-but rather to test targeted hypotheses about what may affect people's behavioral intention-their willingness to participate-as new norms of video usage emerge.…”

mentioning

confidence: 99%

Predictors of willingness to participate in survey interviews conducted by live video.

Schober,

Okon,

Conrad

et al. 2023

Technology, Mind, and Behavior

View full text Add to dashboard Cite

As people increasingly communicate via live video-even more since the COVID-19 pandemic-how willing will they be to participate via video in large-scale standardized surveys that inform public policy, many of which have historically been carried out inperson? This registered report tests three potential predictors of willingness to participate in a live video interview: How (a) easy, (b) useful, and (c) enjoyable respondents find live video to use in other contexts. A potential survey-specific moderator of these effects is also tested: The extent to which respondents report that they would be uncomfortable answering a particular question on a sensitive topic via live video relative to other survey modes. In the study, 598 online U.S. respondents rated their willingness to take part in a hypothetical live video survey that might ask about personal information, in the context of also rating their willingness to take part in four other survey modes, two interviewer-administered (in-person and telephone) and two self-administered (a text-only web survey and a "prerecorded video" web survey in which respondents play videos of interviewers reading questions and then enter answers). Findings demonstrate that willingness to participate in a live video interview is significantly predicted by the extent to which respondents perceive live video as useful and enjoyable in other contexts and by their relative discomfort disclosing in live video versus other modes.

show abstract

A randomized experiment evaluating survey mode effects for video interviewing

Cited by 7 publications

References 36 publications

Interviewer-Observed Paradata in Mixed-Mode and Innovative Data Collection

Interviewer-Observed Paradata in Mixed-Mode and Innovative Data Collection

The Interaction of Survey Mode and Social Desirability in Measuring Behavior during Wartime

Predictors of willingness to participate in survey interviews conducted by live video.

Contact Info

Product

Resources

About