Evaluation of Cnn Architectures for Gait Recognition Based on Optical Flow Maps

Castro, Francisco M.; Marín-Jiménez, Manuel J.; Guil, Nicolás; López-Tapia, Santiago; Blanca, Nicolás Pérez de la

doi:10.23919/biosig.2017.8053503

Cited by 31 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[41,9,42,43]. However, to the extent of our knowledge, only very few works [26,27,28,32] have applied CNN models to the problem of gait recognition using as input low-level features different to binary silhouettes (in contrast to [23]). The great success of the CNN model is in part due to its use on data where the target can be represented through a feature hierarchy of increasing semantic complexity.…”

Section: Cnn Overviewmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

People identification in video based on the way they walk (i.e. gait) is a relevant task in computer vision using a non-invasive approach. Standard and current approaches typically derive gait signatures from sequences of binary energy maps of subjects extracted from images, but this process introduces a large amount of non-stationary noise, thus, conditioning their efficacy. In contrast, in this paper we focus on the raw pixels, or simple functions derived from them, letting advanced learning techniques to extract relevant features. Therefore, we present a comparative study of different Convolutional Neural Network (CNN) architectures on three low-level features (i.e. gray pixels, optical flow channels and depth maps) on two widely-adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to combine the information obtained from each kind of low-level features. Our experimental results suggest that (i ) the use of hand-crafted energy maps (e.g. GEI) is not necessary, since equal or better results can be achieved from the raw pixels; (ii ) the combination of multiple modalities (i.e. gray pixels, optical flow and depth maps) from different CNNs allows to obtain state-of-the-art results on the gait task with an image resolution several times smaller than the previously reported results; and, (iii ) the selection of the architecture is a critical point that can make the difference between state-of-the-art results or poor results. He et al.[10] proposed a new kind of CNN, named ResNet, which has a large number of convolutional layers and 'residual connections' to avoid the vanishing gradient problem.Although several papers can be found for the task of human action recognition using DL techniques, few works apply DL to the problem of gait recognition. In [22], Hossain and Chetty propose the use of Restricted Boltzmann Machines to extract gait features from binary silhouettes, but a very small probe set (i.e. only ten different subjects) were used for validating their approach. A more recent work, [23], uses a random set of binary silhouettes of a sequence to train a CNN that accumulates the calculated features in order to achieve a global representation of the dataset. In [24], raw 2D GEI are employed to train an ensemble of CNN, where a Multilayer Perceptron (MLP) is used as classifier. Similarly, in [25] a multilayer CNN is trained with GEI data. A novel approach based on GEI is developed on [8], where the CNN is trained with pairs of gallery-probe samples and using a distance metric. Castro et al.[26] use optical flow obtained from raw data frames. An in-dept evaluation of different CNN architectures based on optical flow maps is presented in [27]. Finally, in [28] a multitask CNN with a combined loss function with multiple kind of labels is presented.Despite most CNNs are trained with visual data (e.g. images or videos), there are some works that build CNNs for different kinds of data like inertial sensors or human skel...

show abstract

Section: Cnn Overviewmentioning

confidence: 99%

“…Castro et al[26] use optical flow obtained from raw data frames. An in-dept evaluation of different CNN architectures based on optical flow maps is presented in [27]. Finally, in [28] a multitask CNN with a combined loss function with multiple kind of labels is presented.Despite most CNNs are trained with visual data (e.g.…”

mentioning

confidence: 99%

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Castro

Marín-Jiménez

Guil

et al. 2020

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, 37 reflective markers are needed to be attached to the skin of the experimenter and data are collected using six infrared cameras which can restrict the use environment. In [21], a method for recognizing people has been proposed using a convolutional neural network (CNN) [22] for optical flows from video images. In [23], a 1D convolutional neural network is used to classify three axes of acceleration ( x , y , z axis) data acquired using a smartphone into three walking patterns (walking, running, standing).…”

Section: Introductionmentioning

confidence: 99%

Classification of Gait Type Based on Deep Learning Using Various Sensors with Smart Insole

Lee

Choi

2019

Sensors

View full text Add to dashboard Cite

In this paper, we proposed a gait type classification method based on deep learning using a smart insole with various sensor arrays. We measured gait data using a pressure sensor array, an acceleration sensor array, and a gyro sensor array built into a smart insole. Features of gait pattern were then extracted using a deep convolution neural network (DCNN). In order to accomplish this, measurement data of continuous gait cycle were divided into unit steps. Pre-processing of data were then performed to remove noise followed by data normalization. A feature map was then extracted by constructing an independent DCNN for data obtained from each sensor array. Each of the feature maps was then combined to form a fully connected network for gait type classification. Experimental results for seven types of gait (walking, fast walking, running, stair climbing, stair descending, hill climbing, and hill descending) showed that the proposed method provided a high classification rate of more than 90%.

show abstract

“…Recently, more studies on CNN-based gait recognition have been published [15,[32][33][34][35][36][37]. For example, Wu et al [32] used every raw silhouette from each gait sequence as an individual input in their network.…”

Section: Cnn-based Approaches To Gait Recognitionmentioning

confidence: 99%

“…Both of their networks try to perform similarity learning between a probe GEI and a gallery GEI, then tell whether these two GEIs come from the same person or not. In addition to silhouette-based feature GEI, motion features (e.g., optical flow maps) are also used in some approaches [15,37]. All these approaches achieved significant improvements compared with traditional methods.…”

Section: Cnn-based Approaches To Gait Recognitionmentioning

confidence: 99%

Gait Energy Response Functions for Gait Recognition against Various Clothing and Carrying Status

Makihara

et al. 2018

Applied Sciences

View full text Add to dashboard Cite

Silhouette-based gait representations are widely used in the current gait recognition community due to their effectiveness and efficiency, but they are subject to changes in covariate conditions such as clothing and carrying status. Therefore, we propose a gait energy response function (GERF) that transforms a gait energy (i.e., an intensity value) of a silhouette-based gait feature into a value more suitable for handling these covariate conditions. Additionally, since the discrimination capability of gait energies, as well as the degree to which they are affected by the covariate conditions, differs among body parts, we extend the GERF framework to spatially dependent GERF (SD-GERF) which accounts for spatial dependence. Moreover, the proposed GERFs are represented as a vector in the transformation lookup table and are optimized through an efficient generalized eigenvalue problem in a closed form. Finally, two post-processing techniques, Gabor filtering and spatial metric learning, are employed for the transformed gait features to boost the accuracy. Experimental results with three publicly available datasets including clothing and carrying status variations show the state-of-the-art performance of the proposed method compared with other state-of-the-art methods.

show abstract

Evaluation of Cnn Architectures for Gait Recognition Based on Optical Flow Maps

Cited by 31 publications

References 21 publications

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

Classification of Gait Type Based on Deep Learning Using Various Sensors with Smart Insole

Gait Energy Response Functions for Gait Recognition against Various Clothing and Carrying Status

Contact Info

Product

Resources

About