Empirical Evidence for a Diminished Sense of Agency in Speech Interfaces

Abstract

While the technology underlying speech interfaces has improved in recent years, our understanding of the human side of speech interactions remains limited. This paper provides new insight on one important human aspect of speech interactions: the sense of agency -defined as the experience of controlling one's own actions and their outcomes. Two experiments are described. In each case a voice command is compared with keyboard input. Agency is measured using an implicit metric: intentional binding. In both experiments we find that participants' sense of agency is significantly reduced for voice commands as compared to keyboard input. This finding presents a fundamental challenge for the design of effective speech interfaces. We reflect on this finding and, based on current theory in HCI and cognitive neuroscience, offer possible explanations for the reduced sense of agency observed in speech interfaces.

The State of Speech in HCI: Trends, Themes and Challenges

“…Other measures included system recognition of user input (e.g. Oviatt et al, 2008), user agency (Limerick, Moore, & Coyle, 2015), and assessing user creativity and self-disclosure (Wang & Nass, 2005), and cognitive load (Truschin et al, 2014).…”

“…Other data collection methods, which did not fit easily into the other categories, appeared five times in the papers. These included corpus data (Derriks & Willems, 1998), focus groups (Cowan et al, 2017), user modelling (Schaffer et al, 2015), perceived sense of agency (Limerick et al, 2015) and physiological data (Kallinen & Ravaja, 2005). One paper was not explicitly clear in the methods for evaluating the iterative design of a tutor system, though it is assumed some form of interview was used (Hakulinen, Turunen, Salonen, & Räihä, 2004).…”

“…Seven papers explore just speech and keyboard/text input. Two papers report a negative impact of using speech, with speech reducing ease of system use (Begany et al, 2015), and a user's sense of agency (Limerick et al, 2015) compared to more traditional input modalities such as keyboard and mouse. Conversely, two papers report improvements when using speech as an input modality.…”

Effects of Timing on Users’ Agency during Mixed-Initiative Interaction

“…A person will have a sense of agency when they consider themselves to have the ownership of, and be responsible for, the consequences that their actions have in the external world (Coyle et al, 2012;Limerick, Moore and Coyle, 2015). Cognitive neuroscientists take one of two stances in explaining how the sense of agency arises.…”

“…The first can be summarised as how to take actions, and focuses on how different input modalities (e.g. speech, gesture, or skin input) affect users' sense of control (Limerick, Moore and Coyle, 2015;Harrison, Tan and Morris, 2010;Iio et al, 2011;Nash and Blackwell, 2008). A second focuses on how to present consequences, comparing alternative output modalities (e.g.…”