Are you talking to me?

Abstract

In this paper, we present our humanoid robot Meka, participating in a multi party human robot dialogue scenario. Active arbitration of the robot's attention based on multi-modal stimuli is utilised to observe persons which are outside of the robots field of view. We investigate the impact of this attention management and addressee recognition on the robot's capability to distinguish utterances directed at it from communication between humans. Based on the results of a user study, we show that mutual gaze at the end of an utterance, as a means of yielding a turn, is a substantial cue for addressee recognition. Verification of a speaker through the detection of lip movements can be used to further increase precision. Furthermore, we show that even a rather simplistic fusion of gaze and lip movement cues allows a considerable enhancement in addressee estimation, and can be altered to adapt to the requirements of a particular scenario.

To Whom are You Talking? A Deep Learning Model to Endow Social Robots with Addressee Estimation Skills

“…They approached the task as a binary classification (host speaking either to the robot or to the guest) using visual data (automatically extracted head pose) and speech data. In a multiparty HRI scenario, Richter et al [18] opted for a rule-based model taking as input the human's lips movement and the mutual gaze between the human and the robot to understand if an utterance was addressed to the robot or not. After a dataset collection of multi-user human-virtual agent interaction, Huang et al [19] trained an SVM classifier for a binary classification (robot addressed or human partner addressed) by giving as input several features related to prosody, utterance length, and head direction and equipped the virtual agent with a model for realtime Addressee Estimation.…”

Enhancing Human–Robot Collaboration through a Multi-Module Interaction Framework with Sensor Fusion: Object Recognition, Verbal Communication, User of Interest Detection, Gesture and Gaze Recognition

“…In their study, Richter et al [61] proposed lip movement detection to verify the active speaker and suggested that mutual gaze at the end of an utterance is a significant cue for addressee recognition in multi-party HRI scenarios. Meanwhile, Everingham et al [62] used the temporal motion of facial landmarks to detect speech, assuming that motion in the lip area indicates speech.…”

I–C–E Framework: Concepts for Group Dynamics Research in Human-Robot Interaction

“…Especially communication management (e.g., turntaking, back-channeling) and relational communication [12] are heavily based on nonverbal messages and have been addressed in HRI dyads for years. Hence, research groups now shifted to work on realising attention management, turntaking gaze behaviour and other social gaze behaviour for robots in multi-party interactions (e.g., [7,62,75]). Motion in human groups has to be interpreted in real time to anticipate future actions of human group members and synthesize the robots' own motion accordingly [40].…”