Parametric hidden Markov models for gesture recognition

Abstract

ÐA new method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Last, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction.

Recognizing 50 human action categories of web videos

“…Template based methods [1], modeling the dynamics of human motion using finite state models [6] or hidden Markov models [21], and Bag of Features models [10,4,11,22] (BOF) are a few well known approaches taken to solve action recognition. Most of the Datasets Number Camera Background of Actions motion KTH 6 slight motion static Weizmann 10 not present static IXMAS 14 not present static UCF Sports 9 present dynamic HOHA 12 present dynamic UCF11 11 present dynamic UCF50 50 present dynamic HMDB51 51(47) present dynamic Table 1 Action Datasets recent work has been focused on BOF in one form or another.…”

Unstructured human activity detection from RGBD images

“…Early work by Wilson and Bobick [39] extended HMM to parametric HMM for recognizing pointing gestures. Fine et al [8] introduced hierarchical HMM, which was later extended by Bui et al [3] to a general structure in which each child can have multiple parents.…”

Hybrid Dynamical Models of Human Motion for the Recognition of Human Gaits

“…As opposed to standard approaches using discrete models such as Hidden Markov Models (HMMs) and their variants (Oliver et al 2000;Wilson and Bobick 1999), hybrid models capture both the discrete and continuous character of human motion and can be used for both synthesis (Bissacco 2005) and recognition.…”

“…In all cases the first step consists in deriving a compact representation of the motion, such as binary silhouettes (Sarkar et al 2005;Kale et al 2004), optical flow (Little and Boyd 1998), joint angles of an articulated body model with image-based tracking (Bregler 1997;Bissacco et al 2001;North et al 2000), or other spatio-temporal motion descriptors (BenAbdelkader et al 2004;Efros et al 2003;Zelnik-Manor and Irani 2006). Then some statistics are computed on the reduced data and pattern recognition techniques such as principal component analysis (BenAbdelkader et al 2004), bilinear models (Lee and Elgammal 2004), Hidden Markov Models (He and Debrunner 2000;Kale et al 2004;Wilson and Bobick 1999;Oliver et al 2000), KNearest Neighbor classification (Little and Boyd 1998) or Support Vector Machines (Lee and Grimson 2002) are used to solve the classification problem.…”