From handcrafted to learned representations for human action recognition: A survey

Zhen, Xiantong; Shao, Ling; Maybank, Stephen J.; Chellappa, Rama

doi:10.1016/j.imavis.2016.06.007

Cited by 124 publications

(75 citation statements)

References 97 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Each body skeleton consists of 15 joints. The major challenges of this dataset are: (1) in most interactions, one subject is acting, while the other subject is reacting; and (2) the 3D measurement accuracies of the joint coordinates are low in many sequences.…”

Section: Evaluation Datasetsmentioning

confidence: 99%

“…Human action recognition is a fast developing research area due to its wide applications in intelligent surveillance, human-computer interaction, robotics, and so on. In recent years, human activity analysis based on human skeletal data has attracted a lot of attention, and various methods for feature extraction and classifier learning have been developed for skeleton-based action recognition [1], [2], [3]. A hidden Markov model (HMM) is utilized by Xia et al [4] to model the temporal dynamics over a histogram-based representation of joint positions for action recognition.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

Liu

Shahroudy

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

440

244

View full text Add to dashboard Cite

Abstract-Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.

show abstract

Section: Evaluation Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

Liu

Shahroudy

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

440

244

View full text Add to dashboard Cite

show abstract

“…The sparse representation is suitable for the categorization tasks in images and videos. Dictionary learning-based approaches have been employed in wide range of computer vision applications such as image classification and action recognition [123]. The concept of dictionary learning is similar to BoVW model because both are based on the representative vectors learned from the large number of samples.…”

Section: Dictionary Learning-based Approachesmentioning

confidence: 99%

A Comprehensive Review on Handcrafted and Learning-Based Action Representation Approaches for Human Activity Recognition

2017

View full text Add to dashboard Cite

Human activity recognition (HAR) is an important research area in the fields of human perception and computer vision due to its wide range of applications. These applications include: intelligent video surveillance, ambient assisted living, human computer interaction, human-robot interaction, entertainment, and intelligent driving. Recently, with the emergence and successful deployment of deep learning techniques for image classification, researchers have migrated from traditional handcrafting to deep learning techniques for HAR. However, handcrafted representation-based approaches are still widely used due to some bottlenecks such as computational complexity of deep learning techniques for activity recognition. However, approaches based on handcrafted representation are not able to handle complex scenarios due to their limitations and incapability; therefore, resorting to deep learning-based techniques is a natural option. This review paper presents a comprehensive survey of both handcrafted and learning-based action representations, offering comparison, analysis, and discussions on these approaches. In addition to this, the well-known public datasets available for experimentations and important applications of HAR are also presented to provide further insight into the field. This is the first review paper of its kind which presents all these aspects of HAR in a single review article with comprehensive coverage of each part. Finally, the paper is concluded with important discussions and research directions in the domain of HAR.

show abstract

“…There are two major approaches for activity recognition; these include the traditional handcrafted feature-based representation, and learning-based representation. The learning-based representation, and in particular, the deep learning, introduced the concept of endto-end learning by using the trainable feature extractor followed by a trainable classifier [3,4]. The deep leaning based approaches have revealed the remarkable progress for action recognition in videos.…”

Section: Introductionmentioning

confidence: 99%

Human action recognition using transfer learning with deep representations

Sargano

Wang

Angelov

et al. 2017

2017 International Joint Conference on Neural Networks (IJCNN)

130

View full text Add to dashboard Cite

Abstract-Human action recognition is an important research area which has captured lot of attention from the research community due to its significant applications. Recently, due to the popularity and successful implementation of deep learning-based methods for image analysis, object recognition, and speech recognition. Researchers are motivated to shift from traditional feature-based approach to deep learning. This research work presents an innovative method for human action recognition using pre-trained Convolutional Neural Networks (CNNs) model as a source architecture for extracting features from the target dataset, followed by a hybrid Support Vector Machines and K-Nearest Neighbor (SVM-KNN) classifier for action classification. It has been observed that already learnt CNN based representations on large-scale annotated dataset are successfully transferable to action recognition task with limited training dataset. The proposed method is evaluated on two well-known action datasets, i.e., UCF sports and KTH. The comparative analysis suggests that the proposed method is better than handcrafted feature-based methods in terms of accuracy.

show abstract

From handcrafted to learned representations for human action recognition: A survey

Cited by 124 publications

References 97 publications

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

A Comprehensive Review on Handcrafted and Learning-Based Action Representation Approaches for Human Activity Recognition

Human action recognition using transfer learning with deep representations

Contact Info

Product

Resources

About