Static Gesture Recognition Combining Graph and Appearance Features

Abstract

Abstract-In this paper we propose the combination of graphbased characteristics and appearance-based descriptors such as detected edges for modeling static gestures. Initially we convolve the original image with a Gaussian kernel and blur the image. Canny edges are then extracted. The blurring is performed in order to enhance some characteristics in the image that are crucial for the topology of the gesture (especially when the fingers are overlapping). There are a large number of properties that can describe a graph, one of which is the adjacency matrix that describes the topology of the graph itself. We approximate the topology of the hand utilizing Neural Gas with Competitive Hebbian Learning, generating a graph. From the graph we extract the Laplacian matrix and calculate its spectrum. Both canny edges and Laplacian spectrum are used as features. As a classifier we employ Linear Discriminant Analysis with Bayes' Rule. We apply our method on a published American Sign Language dataset (ten classes) and the results are very promising and further study of this approach is imminent from the authors.

Graph Spectral Domain Feature Learning With Application to in-Air Hand-Drawn Number and Shape Recognition

“…Some of the previous work on graph-based hand gesture recognition has been restricted to static hand pose recognition rather than dynamic hand movements [43]- [48]. They are also mainly based on graph theory rather than spectral domain features.…”

“…They are also mainly based on graph theory rather than spectral domain features. A few of the methods include edge-based hand features [43]- [46], tree-based representation [47] and a combination of hand appearance and graph features and graph eigenvalues [48].…”

Feature selection for hand pose recognition in human-robot object exchange scenario

“…This demonstrates the capability of the feature selection system to adapt and choose the best discriminative features depending on the poses to be recognized. The same hand postures of the current ASL database [19] have been classified in [30] using appearance features and a Bayes classifier. Table VIII compares the results of both researches, and it can be noticed that in most of the cases the performance of our investigation overcomes the results achieved in [30].…”

“…The same hand postures of the current ASL database [19] have been classified in [30] using appearance features and a Bayes classifier. Table VIII compares the results of both researches, and it can be noticed that in most of the cases the performance of our investigation overcomes the results achieved in [30]. As both the features and the classifier are different in both researches, it cannot be deduced without more testing whether the feature set or the classifier are responsible for the improvement of the results.…”

Graph spectral domain feature representation for in-air drawn number recognition

“…The available studies of hand gesture recognition based on graphs have been restricted to static hand gestures rather than dynamic hand gestures [14] [15]. They also focused on employing graph theory rather than investigating the spectral graph domain [6] [16].…”