Learning action descriptors for recognition

Marín-Jiménez, Manuel J.; Blanca, Nicolás Pérez de la; Mendoza, M. Ángeles; Lucena, Manuel; Fuertes, José Manuel

doi:10.1109/wiamis.2009.5031418

Cited by 5 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A new realm of the feature learning field for recognition tasks started with the advent of Deep Learning (DL) architectures [11]. These architectures are suitable for discovering good features for classification tasks [12,13]. Recently, DL approaches based on CNN have been used on image-based tasks with great success [9,14,15].…”

Section: Feature Learningmentioning

confidence: 99%

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

People identification in video based on the way they walk (i.e. gait) is a relevant task in computer vision using a non-invasive approach. Standard and current approaches typically derive gait signatures from sequences of binary energy maps of subjects extracted from images, but this process introduces a large amount of non-stationary noise, thus, conditioning their efficacy. In contrast, in this paper we focus on the raw pixels, or simple functions derived from them, letting advanced learning techniques to extract relevant features. Therefore, we present a comparative study of different Convolutional Neural Network (CNN) architectures on three low-level features (i.e. gray pixels, optical flow channels and depth maps) on two widely-adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to combine the information obtained from each kind of low-level features. Our experimental results suggest that (i ) the use of hand-crafted energy maps (e.g. GEI) is not necessary, since equal or better results can be achieved from the raw pixels; (ii ) the combination of multiple modalities (i.e. gray pixels, optical flow and depth maps) from different CNNs allows to obtain state-of-the-art results on the gait task with an image resolution several times smaller than the previously reported results; and, (iii ) the selection of the architecture is a critical point that can make the difference between state-of-the-art results or poor results. He et al.[10] proposed a new kind of CNN, named ResNet, which has a large number of convolutional layers and 'residual connections' to avoid the vanishing gradient problem.Although several papers can be found for the task of human action recognition using DL techniques, few works apply DL to the problem of gait recognition. In [22], Hossain and Chetty propose the use of Restricted Boltzmann Machines to extract gait features from binary silhouettes, but a very small probe set (i.e. only ten different subjects) were used for validating their approach. A more recent work, [23], uses a random set of binary silhouettes of a sequence to train a CNN that accumulates the calculated features in order to achieve a global representation of the dataset. In [24], raw 2D GEI are employed to train an ensemble of CNN, where a Multilayer Perceptron (MLP) is used as classifier. Similarly, in [25] a multilayer CNN is trained with GEI data. A novel approach based on GEI is developed on [8], where the CNN is trained with pairs of gallery-probe samples and using a distance metric. Castro et al.[26] use optical flow obtained from raw data frames. An in-dept evaluation of different CNN architectures based on optical flow maps is presented in [27]. Finally, in [28] a multitask CNN with a combined loss function with multiple kind of labels is presented.Despite most CNNs are trained with visual data (e.g. images or videos), there are some works that build CNNs for different kinds of data like inertial sensors or human skel...

show abstract

Section: Feature Learningmentioning

confidence: 99%

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…A thorough experimental study is carried out on two widely used datasets: ViHASi and Weizmann. In addition, this paper shows that the combination of simple binary silhouettes with aHOF features [12] improves the discrimination in a kNN-based classification framework.…”

Section: Introductionmentioning

confidence: 85%

“…Combining silhouettes and motion. In this experiment, each frame is represented by the vector obtained by concatenating a binary silhouette to an aHOF descriptor [12]. For each frame, we compute its aHOF motion descriptor 2 by using the 20 previous frames.…”

Section: B Experiments On Weizmann Datasetmentioning

confidence: 99%

RBM-based Silhouette Encoding for Human Action Modelling

Marín-Jiménez

Blanca

Mendoza

2010

2010 20th International Conference on Pattern Recognition

View full text Add to dashboard Cite

Abstract-In this paper we evaluate the use of Restricted Bolzmann Machines (RBM) in the context of learning and recognizing human actions. The features used as basis are binary silhouettes of persons. We test the proposed approach on two datasets of human actions where binary silhouettes are available: ViHASi (synthetic data) and Weizmann (real data). In addition, on Weizmann dataset, we combine features based on optical flow with the associated binary silhouettes. The results show that thanks to the use of RBM-based models, very informative and shorter feature vectors can be obtained for the classification tasks, improving the classification performance.

show abstract

“…The Discriminative Restricted Boltzmann Machine (DRBM) has been recently proposed as a state‐of‐the‐art statistical machine learning tool in supervised and semi‐supervised classification settings (Larochelle and Bengio 2008). Until now, empirical evaluation of this algorithm has been mostly limited to image or text data (Larochelle and Bengio 2008; Marin‐Jimenez et al. 2009).…”

Section: Introductionmentioning

confidence: 99%

“…The Discriminative Restricted Boltzmann Machine (DRBM) has been recently proposed as a state-of-the-art statistical machine learning tool in supervised and semi-supervised classification settings (Larochelle and Bengio 2008). Until now, empirical evaluation of this algorithm has been mostly limited to image or text data (Larochelle and Bengio 2008;Marin-Jimenez et al 2009). In this paper, we compare it with the popular Support Vector Machine (SVM) (see Cortes and Vapnik 1995) on a challenging classification task consisting in performing automatic target classification of impulsive sources such as detonations of different kinds of weapons.…”

Section: Introductionmentioning

confidence: 99%

Detonation Classification From Acoustic Signature With the Restricted Boltzmann Machine

Bengio

Chapados

Delalleau

et al. 2012

Computational Intelligence

View full text Add to dashboard Cite

We compare the recently proposed Discriminative Restricted Boltzmann Machine (DRBM) to the classical Support Vector Machine (SVM) on a challenging classification task consisting in identifying weapon classes from audio signals. The three weapon classes considered in this work (mortar, rocket, and rocket‐propelled grenade), are difficult to reliably classify with standard techniques because they tend to have similar acoustic signatures. In addition, specificities of the data available in this study make it challenging to rigorously compare classifiers, and we address methodological issues arising from this situation. Experiments show good classification accuracy that could make these techniques suitable for fielding on autonomous devices. DRBMs appear to yield better accuracy than SVMs, and are less sensitive to the choice of signal preprocessing and model hyperparameters. This last property is especially appealing in such a task where the lack of data makes model validation difficult.

show abstract

Learning action descriptors for recognition

Cited by 5 publications

References 17 publications

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

RBM-based Silhouette Encoding for Human Action Modelling

Detonation Classification From Acoustic Signature With the Restricted Boltzmann Machine

Contact Info

Product

Resources

About