Gesture recognition using hybrid generative-discriminative approach with Fisher Vector

Abstract

Gesture recognition is used for many practical applications such as human-robot interaction, medical rehabilitation and sign language. In this paper, we apply a hybrid generative-discriminative approach by using the Fisher Vector to improve the recognition performance. The strategy is to merge the generative approach of Hidden Markov Model dealing with spatio-temporal motion data with the discriminative approach of Support Vector Machine focusing on the classification task. The motion segments are encoded into HMMs, and each segment is converted to FV, whose elements can be obtained as the derivative of the probability of the segment being generated by the HMMs with respect to their parameters. SVM is subsequently trained by the FVs. An input gesture can be classified to corresponding gesture category by SVM. In the experiments, we test our approach by comparing three HMM chain models and four categorization methods on dataset provided by the ChaLearn Looking at People Challenge 2014 (LAP 2014). The results show that similar gesture patterns are clustered closely in several categories. Our approach based left-to-right HMMs outperforms other gesture recognition methods. More specifically, the hybrid generative-discriminative approach overcomes the standard HMM approach and the generative kernel approach overcomes the generative embedding approach. For these results, our approach is effective to improve the recognition performance.The authors are with the

Classification of Multi-class Daily Human Motion using Discriminative Body Parts and Sentence Descriptions

“…When considering these previous approaches, it can be said that methods that use skeleton features tend to achieve higher classification rates. For example, Goutsu et al (2015) proposed a motion model, in which skeleton features obtained from the whole body are represented as a motion feature and the motion feature is input to the support vector machine (SVM) to determine the motion category. In skeleton-based motion classification, some works have focused on discovering the most discriminative joints of the human body.…”

“…In this paper, we follow a similar approach. Compared with Goutsu et al (2015), our approach weights and integrates motion features obtained from local parts of the human body, focusing on discriminative body parts related to the target motion.…”

“…Additionally, when applied to multi-class motion classification, this approach needs to deepen the hierarchical structure of concept diagram, and thus it becomes more difficult to generate natural sentences because the number of state transitions at branches is increased. (Sun and Nevatia 2013;Goutsu et al 2015) is used to represent a motion feature corresponding to each local part. The motion features of all local parts are weighted and integrated by simultaneously learning parameters using MKL with an SVM.…”

“…For more information about the calculation of these derivatives, refer to Goutsu et al (2015). Given a sequence O i and the set of λ k in each local part, a FV-HMM constructed by combining F S(O i , λ k ) obtained from each abstract motion symbol λ k into a single vector is defined as…”

“…First, the design of the proposed motion model is novel: we propose the weighted integration of motion features by combining Fisher vector parameterized by a hidden Markov model (FV-HMM) with multiple kernel learning (MKL). This model can identify the discriminative parts of the human body related to a target motion, resulting in higher accuracy than the model using skeleton information obtained from the whole body as a motion feature (Goutsu et al 2015). By using this vector combination, we demonstrate that our approach is effective for classifying motions, including human-object interactions with variations in the duration of motions, such as daily human motions.…”