Attentive listening system with backchanneling, response generation and flexible turn-taking

Abstract

Attentive listening systems are designed to let people, especially senior people, keep talking to maintain communication ability and mental health. This paper addresses key components of an attentive listening system which encourages users to talk smoothly. First, we introduce continuous prediction of end-of-utterances and generation of backchannels, rather than generating backchannels after end-point detection of utterances. This improves subjective evaluations of backchannels. Second, we propose an effective statement response mechanism which detects focus words and responds in the form of a question or partial repeat. This can be applied to any statement. Moreover, a flexible turn-taking mechanism is designed which uses backchannels or fillers when the turnswitch is ambiguous. These techniques are integrated into a humanoid robot to conduct attentive listening. We test the feasibility of the system in a pilot experiment and show that it can produce coherent dialogues during conversation.

Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview

“…In this task, ERICA mostly listens to senior people talking about topics such as memorable travels and recent activities [6]. Attentive listening is being recognized as effective for maintaining the communication ability of senior people, and many communicative robots are designed for this task.…”

“…In order to generate backchannels, systems need to predict the timing of backchannels, which has been tackled by many works using prosodic features [13,14]. While Existing backchannel generation systems make a prediction of backchanneling after the end of user utterances segmented by IPUs (inter-pausal units), we implement frame-wise continuous prediction of backchannel timing with a logistic regression that predicts if the system should utter a backchannel within the next 500 milliseconds [6]. We also proposed a prediction model of backchannel form (type) based on both prosodic and linguistic features [15].…”

“…We implemented an attentive listening system with ERICA [6,26]. The aforementioned backchannel generation model is a core component in this system.…”

The Influence of Robot Verbal Support on Human Team Members: Encouraging Outgroup Contributions and Suppressing Ingroup Supportive Behavior

“…In order to increase the quality of communicative interactions and to encourage positive behavior from the humans with which they interact, HRI researchers have incorporated backchanneling behaviors in human-robot interactions (Lala et al, 2017;Ramachandran et al, 2018). For example, Ramachandran et al (2018) designed a tutoring robot to display the non-verbal backchannel of head nodding while a child responded to one of the robot's prompts.…”

Neural Generation of Dialogue Response Timings

“…Models that intentionally generate responsive overlap have been proposed in DeVault et al (2011);Dethlefs et al (2012). While other models have also been proposed that generate appropriate response timings for fillers (Nakanishi et al, 2018;Lala et al, 2019) and backchannels (Morency et al, 2010;Meena et al, 2014;Lala et al, 2017).…”