Profile HMMs for skeleton-based human action recognition

Ding, Wenwen; Fu, Xujia; Cheng, Fei

doi:10.1016/j.image.2016.01.010

Cited by 26 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An LDS was used to learn the temporal evolution of the bioinspired features. Using the motion velocities, direction of the motion, and curvatures of the 3D trajectories, Ding et al [27] categorized the foremost actions into two types of actionunits: dynamic instants and intervals. They utilized selforganizing mapping (SOM) [28] to cluster the action-units with the spatiotemporal feature and employed the sequences of the discrete symbols of each action to build profile hidden Markov models (PHMMs) [29] to obtain the spatiotemporal information between the action-units in the given actions.…”

Section: Related Workmentioning

confidence: 99%

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Cheng

et al. 2019

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

Recently, skeleton-based action recognition has become a very important topic in the field of computer vision. It is a challenging task to accurately build a human action model and precisely distinguish similar human actions. In this paper, an action (skeleton sequence) is represented as a third-order nonnegative tensor time series to capture the original spatiotemporal information of the action. As a linear dynamical system (LDS) is an efficient tool for encoding the spatiotemporal data in various disciplines, this paper proposes a nonnegative tensor-based LDS (nLDS) to model the third-order nonnegative tensor time series. Nonnegative Tucker decomposition (NTD) is utilized to estimate the parameters of the nLDS model. These parameters are used to build extended observability sequence O∞T for the action, which implies that O∞T can be considered as the feature descriptor of the action. To avoid the limitations introduced by approximating O∞T with a finite-order matrix, we represent an action as a point on infinite Grassmann manifold comprising the orthonormalized extended observability sequences. The classification task can be performed by dictionary learning and sparse coding on the infinite Grassmann manifold. The experimental results on the MSR-Action3D, UTKinect-Action, and G3D-Gaming datasets demonstrate that the proposed approach achieves a better performance in comparison with the state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Cheng

et al. 2019

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, their method does not eliminate noisy skeletons from an action, and it is insufficient to estimate the approximation of an extended observability sequence with a finite Grassmann manifold. Ding et al divided actions into subactions and used the profile hidden Markov model(HMM) to align them [13]. Although their approach accurately extracts the spatial features of an action, it does not solve the following two problems: eliminating noisy skeletons and reducing the time complexity of the profile HMMs.…”

Section: Related Workmentioning

confidence: 99%

“…Different from previous orientationbased approaches, in this study, a rigid-body orientation is represented as six rotation matrices, and each rotation matrix is represented as a point in (3). (2) Traditional approaches based on Lie groups [5,13,14] only consider the spatial information of a skeleton but ignore the temporal information between different skeletons. Therefore, our approach employs the rigid-body motions between different skeletons to describe the temporal variation.…”

Section: Introductionmentioning

confidence: 99%

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Cheng

et al. 2018

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

The human skeleton can be considered as a tree system of rigid bodies connected by bone joints. In recent researches, substantial progress has been made in both theories and experiments on skeleton-based action recognition. However, it is challenging to accurately represent the skeleton and precisely eliminate noisy skeletons from the action sequence. This paper proposes a novel skeletal representation, which is composed of two subfeatures to recognize human action: static features and dynamic features. First, to avoid scale variations from subject to subject, the orientations of the rigid bodies in a skeleton are employed to capture the scale-invariant spatial information of the skeleton. The static feature of the skeleton is defined as a combination of the orientations. Unlike previous orientation-based representations, the orientation of a rigid body in the skeleton is defined as the rotations between the rigid body and the coordinate axes in three-dimensional space. Each rotation is mapped to the special orthogonal group SO(3). Next, the rigid-body motions between the skeleton and its previous skeletons are utilized to capture the temporal information of the skeleton. The dynamic feature of the skeleton is defined as a combination of the motions. Similarly, the motions are represented as points in the special Euclidean group SE(3). Therefore, the proposed skeleton representation lies in the Lie group (SE(3)×⋯×SE(3), SO(3)×⋯×SO(3)), which is a manifold. Using the proposed representation, an action can be considered as a series of points in this Lie group. Then, to recognize human action more accurately, a new pattern-growth algorithm named MinP-PrefixSpan is proposed to mine the key-skeleton-patterns from the training dataset. Because the algorithm reduces the number of new patterns in each growth step, it is more efficient than the PrefixSpan algorithm. Finally, the key-skeleton-patterns are used to discover the most informative skeleton sequences of each action (skeleton sequence). Our approach achieves accuracies of 94.70%, 98.87%, and 95.01% on three action datasets, outperforming other relative action recognition approaches, including LieNet, Lie group, Grassmann manifold, and Graph-based model.

show abstract

“…Human gesture recognition is mainly performed on RGB-D (Red, Green, Blue and Depth) data [9], [10], [11], [12], [6], [13] or on skeleton data [14], [15], [16], [17], [18], [13], [19], [20], [21], where skeletal data can be extracted from RGB-D data. To recognize static gestures (i.e.…”

Section: Movementioning

confidence: 99%

Human gesture classification by brute-force machine learning for exergaming in physiotherapy

Deboeverie

Roegiers

Allebosch

et al. 2016

2016 IEEE Conference on Computational Intelligence and Games (CIG)

View full text Add to dashboard Cite

Abstract-In this paper, a novel approach for human gesture classification on skeletal data is proposed for the application of exergaming in physiotherapy. Unlike existing methods, we propose to use a general classifier like Random Forests to recognize dynamic gestures. The temporal dimension is handled afterwards by majority voting in a sliding window over the consecutive predictions of the classifier. The gestures can have partially similar postures, such that the classifier will decide on the dissimilar postures. This brute-force classification strategy is permitted, because dynamic human gestures show sufficient dissimilar postures. Online continuous human gesture recognition can classify dynamic gestures in an early stage, which is a crucial advantage when controlling a game by automatic gesture recognition. Also, ground truth can be easily obtained, since all postures in a gesture get the same label, without any discretization into consecutive postures. This way, new gestures can be easily added, which is advantageous in adaptive game development. We evaluate our strategy by a leave-one-subject-out cross-validation on a self-captured stealth game gesture dataset and the publicly available Microsoft Research Cambridge-12 Kinect (MSRC-12) dataset. On the first dataset we achieve an excellent accuracy rate of 96.72%. Furthermore, we show that Random Forests perform better than Support Vector Machines. On the second dataset we achieve an accuracy rate of 98.37%, which is on average 3.57% better then existing methods.

show abstract

Profile HMMs for skeleton-based human action recognition

Cited by 26 publications

References 25 publications

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Human gesture classification by brute-force machine learning for exergaming in physiotherapy

Contact Info

Product

Resources

About