Engaging human-to-robot attention using conversational gestures and lip-synchronization

Abstract

Abstract-Human-Robot Interaction (HRI) is one of the most important subfields of social robotics. In several applications, text-to-speech (TTS) techniques are used by robots to provide feedback to humans. In this respect, a natural synchronization between the synthetic voice and the mouth of the robot could contribute to improve the interaction experience. This paper presents an algorithm for synchronizing Text-To-Speech systems with robotic mouths. The proposed approach estimates the appropriate aperture of the mouth based on the entropy of the synthetic audio stream provided by the TTS system. The paper also describes the cost-efficient robotic head which has been used in the experiments and introduces the use of conversational gestures for engaging Human-Robot Interaction. The system, which has been implemented in C++ and can perform in realtime, is freely available as part of the RoboComp open-source robotics framework. Finally, the paper presents the results of the opinion poll that has been conducted in order to evaluate the interaction experience.

A Novel Speech to Mouth Articulation System for Realistic Humanoid Robots

“…However, the study details the limitations of the lip-synchronisation application when decoding live input as the system neglected to recognise the variability in the tonality and pitch of different human voices which frequently produced incorrect lip positions. A similar study [25] explored the implementation of speech wave signal processing to create jaw movement in robots, derived from previous studies by [26] and [27]. Although the mouth articulation system was successful in correctly analysing incoming audio to detect frequency on/off status for synchronisation with the open/closed mouth positions of the robots, this study and the previous examples neglect lip synchronisation and focus solely on jaw position to incoming sound frequencies.…”