Stereovision-Only Based Interactive Mobile Robot for Human-Robot Face-to-Face Interaction

Abstract

In this paper, we present a stereovision-only based interactive mobile robot for supporting human-robot face-to-face interaction in the real world. A three-level architecture, which consists of sensor level, perception level and behavior level, is designed for the robot in order to perceive, understand and react to the human activity during interaction based only on visual information. A high performance stand-alone stereovision system (RGBD imager), developed in our lab, is applied to obtain the composite of color (RGB) images and dense disparity (D) maps at video rate. The RGBD imager allows the robot a human-like 3-D visual perception ability to (1) autonomously detect the human of interest whom the robot could interact with using the offline learning approaches, and (2) focus exclusively on the target human while both the human and the robot are moving during interaction using on-line learning approaches. We demonstrate and evaluate the performance of our interactive mobile robot in an office environment. The experimental results show that a reliable and dynamic face-to-face interaction is achieved, so that the target human face is always kept in the field of view and at a suitable social distance from the robot.

Stereo Vision Based Robot for Remote Monitoring with VR Support

“…Lei Chen [1] developed a stereovision only based interactive mobile robot for HRI application. It uses RGBD sensors designed for this application on a Pioneer 3-AT all-terrain robot platform.…”

“…Use of deep learning can make this robot more intelligent and environment-aware (Fig.2). It provides us with a base for HRI (Human-Robot Interaction) [6] [1] where it senses and interacts with its environment or transfers one"s interaction remotely. This robot uses deep neural networks to find and track objects and faces like a human and stream it to the user where the user will able to monitor things effortlessly with 3D experience using VR [9] from remote ( Fig.3).…”