Comprehension and engagement in survey interviews with virtual agents

Abstract

This study investigates how an onscreen virtual agent's dialog capability and facial animation affect survey respondents' comprehension and engagement in “face-to-face” interviews, using questions from US government surveys whose results have far-reaching impact on national policies. In the study, 73 laboratory participants were randomly assigned to respond in one of four interviewing conditions, in which the virtual agent had either high or low dialog capability (implemented through Wizard of Oz) and high or low facial animation, based on motion capture from a human interviewer. Respondents, whose faces were visible to the Wizard (and videorecorded) during the interviews, answered 12 questions about housing, employment, and purchases on the basis of fictional scenarios designed to allow measurement of comprehension accuracy, defined as the fit between responses and US government definitions. Respondents answered more accurately with the high-dialog-capability agents, requesting clarification more often particularly for ambiguous scenarios; and they generally treated the high-dialog-capability interviewers more socially, looking at the interviewer more and judging high-dialog-capability agents as more personal and less distant. Greater interviewer facial animation did not affect response accuracy, but it led to more displays of engagement—acknowledgments (verbal and visual) and smiles—and to the virtual interviewer's being rated as less natural. The pattern of results suggests that a virtual agent's dialog capability and facial animation differently affect survey respondents' experience of interviews, behavioral displays, and comprehension, and thus the accuracy of their responses. The pattern of results also suggests design considerations for building survey interviewing agents, which may differ depending on the kinds of survey questions (sensitive or not) that are asked.

Mimicry and expressiveness of an ECA in human-agent interaction: familiarity breeds content!

“…The benefits for comprehension, according to the authors, are related to gestures and facial expressions affecting redundancy, which results in more learning. Conrad et al (2015) manipulated an agent’s dialog capability (high, low) and facial animation (high, low) and examined their effect on users’ comprehension and engagement in completing a survey. Answers to the survey were more accurate in response to agents with high capability dialog and these agents were judged as more personal and less distant.…”

“…Cue redundancy serves to reduce cognitive load (Stevens et al 2013). Conrad et al (2015) reported differential effects of facial animation and dialog capability moderated by context. The present results further corroborate such a relationship and point to the need to design agents and features in tandem with context and content.…”

Can Conversational Interviewing Improve Survey Response Quality Without Increasing Interviewer Effects?

“…The survey interviewing literature includes several studies demonstrating that CI reduces the bias in survey responses relative to SI (Schober and Conrad, 1997;Conrad and Schober, 2000;Schober et al, 2004Schober et al, , 2012Hubbard et al, 2012;Conrad et al, 2015;Bruckmeier et al, 2015). These studies have shown that, although CI can lead to longer interviews (potentially increasing costs), it can also improve respondents' comprehension of terms in survey questions that may be ambiguous with respect to the situations of particular respondents.…”

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

“…For example, Conrad et al (2015) showed that small talk in survey interviewing to increase the users' comprehension and engagement. Bickmore and Cassell (2005) also used small talk to increase trust.…”