Natural deictic communication with humanoid robots

Abstract

Abstract-A simple view of deictic communication only includes the indication process and recognition process: a person points at an object and says something about it such as "look at this," and then the other person recognizes the pointing gesture and pays attention to the indicated object. However, this simple view lacks three important processes: attention synchronization, context focus, and believability establishment. We refer to these three processes as "facilitation processes" and implement them in a humanoid robot with a motion capturing system. An experiment with 30 subjects revealed that the facilitation processes make deictic communication natural.I. INTRODUCTION The aim of our research is to develop "communication robots" that naturally interact with humans and support daily human activities based on advanced interaction capabilities with their human-like bodies. Since the target audience of a communication robot is ordinary people who do not have specialized computing and engineering knowledge, a conversational interface using both verbal and non-verbal expressions is becoming more important. For example, the precedent research works have shown the effectiveness of facial expressions [1], eye-gaze [2], and gestures [3].This study focuses on deictic communication as shown in Fig. 1. In casual communication, people often use reference terms in combination with pointing, such as saying "look at this" while pointing at an object in order to draw others' attention to the object. A simple view of deictic communication is the use of attention-drawing behavior based on pointing gestures with reference terms. People use pointing gestures as one of the mechanisms of joint attention [4]. Reference terms are important for quickly and naturally informing the listener of an indicated object's location. It is important for a communication robot to be capable of both understanding people's attention-drawing behavior and performing attention-drawing behavior for people [5]. This mutual attention-drawing activity is classified into an indication process and a recognition process in Fig. 1.However, this simple view only encompasses the processes related to the interpretation of information, and it lacks a process for facilitating interaction. We believe that three major processes facilitate deictic communication. The first one is attention synchronization. When we point at anAll the author is with ATR Intelligent Robotics and Communication Laboratory, 2-2-2 Hikaridai, Keihanna Science City, Kyoto, Japan (phone:+81-774-95-1492, e-mail: kanda@atr.jp). Osamu Sugiyama and Michita Imai are also with graduate school, Keio University, Kouhokuku, Yokohama City, Kanagawa, Japan. Hiroshi Ishiguro is with Faculty of Engineering, Osaka University, Suita City, Osaka, Japan. object, we pay attention to whether the listener's gaze is following our pointing gesture. Thus, if a communication robot is listening to a person using attention-drawing behavior, it needs to indicate that it is paying attention to the pointing gesture in or...

Modeling and Evaluating Narrative Gestures for Humanlike Robots

“…While recognition of human gestures has long been a research topic in robotics and computer vision, recent research has explored how gesture recognition might facilitate human-robot interaction [6,24,35].…”

“…Research on realizing gestures for robots has been focused primarily on a subset of the four typical types of gestures, such as how robots might use deictic gestures to give directions to their users [25] or to learn a task from their users [35]. This line of research also includes novel approaches to and control architectures for generating gestures [4,23,30].…”

Point-and-Command Paradigm for Interaction with Assistive Robots

“…Other works try to enable a robot to generate its own pointing gestures [18] [24], or investigate the use of pointing gestures in interactions between a human and a robot [25] or between two robots [26], but most papers focus on the ability of a robot to respond to pointing gestures performed by human operators. Some papers focus on the recognition methodology and the estimation of pointing direction [27] [28]; others focus on the control component of the system [29] or on the process of achieving a natural deictic communication between robots and humans [30].…”

Language Use in Joint Action: The Means of Referring Expressions

“…Sato et al [13] also discussed the usefulness of deictic gestures in a human-robot interaction. Using a highly specified situation, Sugiyama et al [14] showed that the deictic referring (i.e., the use of eye fixation and demonstratives) could have been more effective than the symbolic referring (i.e., the use of numbers) in a human-robot interaction.…”

“…Demonstratives such as "this" and "that" are a unique class of deictic words that play an important role in joint action [14]. When a person says, "I want to buy this clock," he/she obviously intends to buy a specific clock and believes that the addressee would know the correct referent compared to her saying "I want to buy a clock."…”