RBM-based Silhouette Encoding for Human Action Modelling

Marín-Jiménez, Manuel J.; Blanca, Nicolás Pérez de la; Mendoza, M. Ángeles

doi:10.1109/icpr.2010.245

Cited by 2 publications

(3 citation statements)

References 17 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A new realm of the feature learning field for recognition tasks started with the advent of Deep Learning (DL) architectures [11]. These architectures are suitable for discovering good features for classification tasks [12,13]. Recently, DL approaches based on CNN have been used on image-based tasks with great success [9,14,15].…”

Section: Feature Learningmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

People identification in video based on the way they walk (i.e. gait) is a relevant task in computer vision using a non-invasive approach. Standard and current approaches typically derive gait signatures from sequences of binary energy maps of subjects extracted from images, but this process introduces a large amount of non-stationary noise, thus, conditioning their efficacy. In contrast, in this paper we focus on the raw pixels, or simple functions derived from them, letting advanced learning techniques to extract relevant features. Therefore, we present a comparative study of different Convolutional Neural Network (CNN) architectures on three low-level features (i.e. gray pixels, optical flow channels and depth maps) on two widely-adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to combine the information obtained from each kind of low-level features. Our experimental results suggest that (i ) the use of hand-crafted energy maps (e.g. GEI) is not necessary, since equal or better results can be achieved from the raw pixels; (ii ) the combination of multiple modalities (i.e. gray pixels, optical flow and depth maps) from different CNNs allows to obtain state-of-the-art results on the gait task with an image resolution several times smaller than the previously reported results; and, (iii ) the selection of the architecture is a critical point that can make the difference between state-of-the-art results or poor results. He et al.[10] proposed a new kind of CNN, named ResNet, which has a large number of convolutional layers and 'residual connections' to avoid the vanishing gradient problem.Although several papers can be found for the task of human action recognition using DL techniques, few works apply DL to the problem of gait recognition. In [22], Hossain and Chetty propose the use of Restricted Boltzmann Machines to extract gait features from binary silhouettes, but a very small probe set (i.e. only ten different subjects) were used for validating their approach. A more recent work, [23], uses a random set of binary silhouettes of a sequence to train a CNN that accumulates the calculated features in order to achieve a global representation of the dataset. In [24], raw 2D GEI are employed to train an ensemble of CNN, where a Multilayer Perceptron (MLP) is used as classifier. Similarly, in [25] a multilayer CNN is trained with GEI data. A novel approach based on GEI is developed on [8], where the CNN is trained with pairs of gallery-probe samples and using a distance metric. Castro et al.[26] use optical flow obtained from raw data frames. An in-dept evaluation of different CNN architectures based on optical flow maps is presented in [27]. Finally, in [28] a multitask CNN with a combined loss function with multiple kind of labels is presented.Despite most CNNs are trained with visual data (e.g. images or videos), there are some works that build CNNs for different kinds of data like inertial sensors or human skel...

show abstract

Section: Feature Learningmentioning

confidence: 99%

“…2 we show five frames distributed every six frames along a subsequence of twenty-five frames in total (i.e. frames 1,7,13,19,25). The first row shows the horizontal component of the OF (x-axis displacement) and second row shows the vertical component of the OF (y-axis displacement).…”

Section: Optical Flowmentioning

confidence: 99%

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Though advances have recently been made for handwritten digits [15] and human silhouettes [16], the task of learning accurate distributions of holistic shape in general still remains a challenging and open problem. One natural approach to this problem is the parts-based one in which the variability of the object's shape is reasoned about in terms of the relationships of its constituent parts [3].…”

Section: Introductionmentioning

confidence: 99%

Factored Shapes and Appearances for Parts-based Object Understanding

Eslami¹,

Williams²

2011

Procedings of the British Machine Vision Conference 2011

View full text Add to dashboard Cite

We present a novel generative framework for learning parts-based representations of object classes. Our model, Factored Shapes and Appearances (FSA), employs a highly factored representation to reason about appearance and shape variability across datasets of images. We propose Markov Chain Monte Carlo sampling schemes for efficient inference and learning, and evaluate the model on a number of datasets. Here we consider datasets that exhibit large amounts of variability, both in the shapes of objects in the scene, and in their appearances. We show that the FSA model extracts meaningful parts from training data, and that its parameters and representation can be used to perform a range of tasks, including object parsing, segmentation and fine-grained categorisation.

show abstract

RBM-based Silhouette Encoding for Human Action Modelling

Cited by 2 publications

References 17 publications

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Factored Shapes and Appearances for Parts-based Object Understanding

Contact Info

Product

Resources

About