An Honest Conversation: Transparently Combining Machine and Human Speech Assistance in Public Spaces

Abstract

There is widespread concern over the ways speech assistant providers currently use humans to listen to users' queries without their knowledge. We report two iterations of the Talk-Back smart speaker, which transparently combines machine and human assistance. In the first, we created a prototype to investigate whether people would choose to forward their questions to a human answerer if the machine was unable to help. Longitudinal deployment revealed that most users would do so when given the explicit choice. In the second iteration we extended the prototype to draw upon spoken answers from previous deployments, combining machine efficiency with human richness. Deployment of this second iteration shows that this corpus can help provide relevant, human-created instant responses. We distil lessons learned for those developing conversational agents or other AI-infused systems about how to appropriately enlist human-in-the-loop information services to benefit users, task workers and system performance.

CUI@CSCW: Collaborating through Conversational User Interfaces

“…This one-day workshop will examine the design and use of Conversational User Interfaces (CUIs) for collaboration. Research on CUIs has been reignited across disciplines by the recent surge in availability of commercially-available systems on smartphones as voice assistants [16], websites as chatbots [8], and as smart speakers in homes [15] and public spaces ranging from slums [18] to museums [2]. These systems are being used for a wide range of purposes and are enabling technology to become accessible to wider audiences including, but by no means not limited to, older adults [21], children [20], non-native speakers [23,24], and the visually impaired [19].…”

“…With devices such as IPAs often being deployed in social and public spaces (e.g. [15,16,18]), multi-party and mixed agent-user conversations, often involving voice, require further understanding to successfully support collaborative working.…”