A Kinect-Based Gesture Recognition Approach for a Natural Human Robot Interface

Abstract

In this paper, we present a gesture recognition system for the development of a human-robot interaction (HRI) interface. Kinect cameras and the OpenNI framework are used to obtain real-time tracking of a human skeleton. Ten different gestures, performed by different persons, are defined. Quaternions of joint angles are first used as robust and significant features. Next, neural network (NN) classifiers are trained to recognize the different gestures. This work deals with different challenging tasks, such as the real-time implementation of a gesture recognition system and the temporal resolution of gestures. The HRI interface developed in this work includes three Kinect cameras placed at different locations in an indoor environment and an autonomous mobile robot that can be remotely controlled by one operator standing in front of one of the Kinects. Moreover, the system is supplied with a people reidentification module which guarantees that only one person at a time has control of the robot. The system's performance is first validated offline, and then online experiments are carried out, proving the real-time operation of the system as required by a HRI interface.

Gesture recognition for humanoid robot teleoperation

“…Weighted Dynamic Time Warping method has been applied to the dynamic gesture recognition. In [10] they recognized ten gestures chosen from the army visual signals to control a mobile robot. Gesture recognition was performed by neural network (NN) classifiers.…”

A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

“…Similar human-robot interaction (HRI) settings are found in the literature, such as [2], where the authors have used Kinect for the detection of pointed direction by the user and for navigation of the robot. In [3], the authors have used multiple Kinects in a fixed workspace and have used neural networks to detect dynamic gestures. Kinect based object recognition through 3D gestures is proposed in [4].…”

Multiple Kinect Sensor Fusion for Human Skeleton Tracking Using Kalman Filtering

“…For example, human activity recognition based on skeleton tracking [26] might be a fundamental component for human robot interaction [24,23], whereby humans and robots can cooperate in tasks in industrial environments based on human skeleton tracking [21]. Transferring human skills and teleoperation of human motions in robots require 3D body/hand poses [22,25].…”