Real time interaction with mobile robots using hand gestures

Abstract

We developed a robust real time hand gesture based interaction system to effectively communicate with a mobile robot which can operate in an outdoor environment. The system enables the user to operate a mobile robot using hand gesture based commands. In particular the system offers direct on site interaction providing better perception of environment to the user. To overcome the illumination challenges in outdoors, the system operates on depth images. Processed depth images are given as input to a convolutional neural network which is trained to detect static hand gestures. The system is evaluated in real world experiments on a mobile robot to show the operational efficiency in outdoor environment

Multi-sensor system for driver's hand-gesture recognition

“…To identify the gesture type, sequences of features for dynamic gestures are used to train classifiers, such as Hidden Markov Models (HMM) [6], conditional random fields [22], Support Vector Machines (SVM) [23], or decision forests. Convolutional DNNs have also been employed previously to detect and recognize 20 gestures from the Italian sign language using RGB-D images of hand regions along with upper-body skeletal features [20], and for classifying 6 static hand gestures using depth images [8]. These previous DNN-based gesture recognition methods are different from our proposed work in their data fusion strategies, features employed, and application scenarios.…”

“…With the availability of cheap consumer depth cameras, gesture recognition systems using depth cameras have also been introduced [7]. With the exception of a few previous methods [8], [9], most vision-based gesture recognition systems have been developed for environments with controlled illumination [10]. The interior of a car is a challenging environment because the lighting conditions vary a lot.…”

Context aware 3D gesture recognition for games and virtual reality

“…Examples include neural networks [Konda et al 2012;Murakami and Taguchi 1991], template matching [Li and Jarvis 2009;Liu and Fujimura 2004], finite state machines [Hong et al 2000;Li and Jarvis 2009], and using the AdaBoost framework [Hoffman et al 2010]. To cover all of them in detail would go beyond the scope of these notes.…”

Hand Gesture Recognition Using Appearance Features Based on 3D Point Cloud

“…Depth information has long been regarded as an essential part of successful gesture recognition [5]. Many researches [6]- [8] extract different features from the depth data, then various classifiers are employed for gesture recognition. These methods all get good effect, but they have to collect a large number of training samples.…”