View and Style-Independent Action Manifolds for Human Activity Recognition

Abstract

Abstract. We introduce a novel approach to automatically learn intuitive and compact descriptors of human body motions for activity recognition. Each action descriptor is produced, first, by applying Temporal Laplacian Eigenmaps to view-dependent videos in order to produce a stylistic invariant embedded manifold for each view separately. Then, all view-dependent manifolds are automatically combined to discover a unified representation which model in a single three dimensional space an action independently from style and viewpoint. In addition, a bidirectional nonlinear mapping function is incorporated to allow projecting actions between original and embedded spaces. The proposed framework is evaluated on a real and challenging dataset (IXMAS), which is composed of a variety of actions seen from arbitrary viewpoints. Experimental results demonstrate robustness against style and view variation and match the most accurate action recognition method.

Probabilistic Feature Extraction from Multivariate Time Series Using Spatio-Temporal Constraints

“…Although different approaches may use slightly different experimental settings, table 1 shows that our framework produces the best performances. In particular, it improves the accuracy of the standard framework [7]. The confusion matrix of recognition for the 'all-view' experiment reveals that our framework performed better when dealing with motions involving the whole body, i.e.…”

“…Here , ( ) returns the centre point of , . The neighbourhood is determined automatically using either dynamic time warping [8] or motion pattern detection [7]. Neighbourhood connections defined in the Laplacian graphs implicitly impose points closeness in the latent space.…”

“…We demonstrate the effectiveness of the novel methodology in a realistic computer vision application by integrating ST-GPLVM within the view independent human action recognition framework proposed in [7]. Here, the training data comprises of time series of action images obtained from evenly spaced views located around the vertical axis of a subject.…”

“…First, for each view (z=1..Z ), silhouettes are extracted from videos, normalized and represented as 3364-dimensional vectors of local space-time saliency features [7]. Then, spatio-temporal constraints are calculated.…”

“…Then, spatio-temporal constraints are calculated. During initialisation, style invariant one-dimensional action manifolds are obtained using temporal LE [8] and subsequently all these view-dependent models are combined to generate a coherent view invariant representation of the action [7]. The outcome of this procedure reveals a torus-like structure which is used to initialise GPLVM and encapsulates both style and view.…”