View-invariant dynamic texture recognition using a bag of dynamical systems

Abstract

In this paper, we consider the problem of categorizing videos of dynamic textures under varying view-point. We propose to model each video with a collection of Linear Dynamics Systems (LDSs) describing the dynamics of spatiotemporal video patches. This bag of systems (BoS) representation is analogous to the bag of features (BoF) representation, except that we use LDSs as feature descriptors. This poses several technical challenges to the BoF framework. Most notably, LDSs do not live in a Euclidean space, hence novel methods for clustering LDSs and computing codewords of LDSs need to be developed. Our framework makes use of nonlinear dimensionality reduction and clustering techniques combined with the Martin distance for LDSs for tackling these issues. Our experiments show that our BoS approach can be used for recognizing dynamic textures in challenging scenarios, which could not be handled by existing dynamic texture recognition methods.

Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

“…To recognize dynamic features, Ravichandran et al [17] model each video with a collection of motion primitives that describe the dynamics of spatiotemporal video patches. This bag-of-motion-primitives (BoMP) approach is similar to the work on using the bag-of-features (BoF) for object recognition which categorizes images by observing the distribution of a small collection of features.…”

Dynamic texture classification using dynamic fractal analysis

“…The generative methods [2,8,12,21,22,24,28] attempt to quantitatively model the underlying physical dynamic system that generates DT sequences and classify DT sequences based on the system parameters of the corresponding physical model. For example, in [24], each pixel is expressed as a linear combination of the neighboring pixels in the spatio-temporal domain.…”

“…Alternatively, discriminative methods [21,27,31] have been proposed for DT classification without explicitly modeling the underlying dynamic system. In [27] spatiotemporal filters are constructed specifically tuned up for certain local DT structures with a few image patterns and motion patterns.…”

“…The descriptor proposed in [31] extends the local binary pattern (LBP) of 2D image to the 3D spatialtemporal volume. [21] combined local LDS model with the bag-of-words model. Compared to generative methods, discriminative methods tend to perform better in the task of DT classification, as shown in experiments of some recent works.…”

“…The UCLA dynamic texture dataset has been widely used in many previous studies [4,6,11,21,22]. It originally contains 50 DT classes, each with four grayscale video sequences captured from different viewpoints.…”