Informative joints based human action recognition using skeleton contexts

Jiang, Min; Kong, Jun; Bebis, George; Huo, Hongtao

doi:10.1016/j.image.2015.02.004

Cited by 67 publications

(51 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several previous works [13], [50] have shown that in each action sequence, there is often a subset of informative joints which are important as they contribute much more to action analysis, while the remaining ones may be irrelevant (or even noisy) for this action. As a result, to obtain a high accuracy of action recognition, we need to identify the informative skeletal joints and concentrate more on their features, meanwhile trying to ignore the features of the irrelevant ones, i.e., selectively focusing (attention) on the informative joints is useful for human action recognition.…”

Section: B Global Context-aware Attention Lstmmentioning

confidence: 99%

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks

Liu

Wang

Duan³

et al. 2018

IEEE Trans. on Image Process.

516

232

View full text Add to dashboard Cite

Human action recognition in 3D skeleton sequences has attracted a lot of research attention. Recently, long short-term memory (LSTM) networks have shown promising performance in this task due to their strengths in modeling the dependencies and dynamics in sequential data. As not all skeletal joints are informative for action recognition, and the irrelevant joints often bring noise which can degrade the performance, we need to pay more attention to the informative ones. However, the original LSTM network does not have explicit attention ability. In this paper, we propose a new class of LSTM network, global context-aware attention LSTM, for skeleton-based action recognition, which is capable of selectively focusing on the informative joints in each frame by using a global context memory cell. To further improve the attention capability, we also introduce a recurrent attention mechanism, with which the attention performance of our network can be enhanced progressively. Besides, a two-stream framework, which leverages coarse-grained attention and fine-grained attention, is also introduced. The proposed method achieves state-of-the-art performance on five challenging datasets for skeleton-based action recognition.

show abstract

Section: B Global Context-aware Attention Lstmmentioning

confidence: 99%

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks

Liu

Wang

Duan³

et al. 2018

IEEE Trans. on Image Process.

516

232

View full text Add to dashboard Cite

show abstract

“…There are only a few attempts in previous works to consider the effect from view variations. A general treatment employs a pre-processing step to transform the 3D joint coordinates from the camera coordinate system to a personcentric coordinate system by placing the body center at the origin, followed by rotating the skeleton such that the body plane is parallel to the (x, y)-plane, to make the skeleton data invariant to absolute location, and the body orientation [45,39,5,51,18,31,24,35]. Such a pre-processing gains partial view-invariant.…”

Section: Introductionmentioning

confidence: 99%

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Zhang

Lan

Xing

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

590

402

View full text Add to dashboard Cite

Skeleton-based human action recognition has recently attracted increasing attention due to the popularity of 3D skeleton data. One main challenge lies in the large view variations in captured human actions. We propose a novel view adaptation scheme to automatically regulate observation viewpoints during the occurrence of an action. Rather than re-positioning the skeletons based on a human defined prior criterion, we design a view adaptive recurrent neural network (RNN) with LSTM architecture, which enables the network itself to adapt to the most suitable observation viewpoints from end to end. Extensive experiment analyses show that the proposed view adaptive RNN model strives to (1) transform the skeletons of various views to much more consistent viewpoints and (2) maintain the continuity of the action rather than transforming every frame to the same position with the same body orientation. Our model achieves significant improvement over the state-of-the-art approaches on three benchmark datasets.

show abstract

“…Even though the depth cameras have generally a better quality of 3D action data than those estimated from monocular video sensors, adopting the 3D joint positions for human-object interaction is not sufficient to classify actions that 35 includes interaction with objects. During a human object interaction scene, the hands may hold objects thus are hardly detected or recognized due to heavy occlusions and appearance variations [13].…”

mentioning

confidence: 99%

“…Recently, with the development of the commodity depth sensors like Kinect, there has been a lot of interests in human action recognition from depth data such as [9], [8], [10], [11], [12], [34], [35], [36]. Instead of covering all ideas exhaustively, we direct interested readers to some recent surveys [7], [37], [38],…”

mentioning

confidence: 99%

Distances evolution analysis for online and off-line human object interaction recognition

Meng

Drira

Boonært

2018

Image and Vision Computing

View full text Add to dashboard Cite

Human action recognition in 3D sequences is one of the most challenging and active areas of research in the computer vision domain. However designing automatic systems that are robust to significant variability due to object combinations and high complexity of human motions are more challenging in addition to the typical requirements such as rotation, translation, and scale invariance is challenging task. In this paper, we propose a spatio-temporal modeling of human-object interaction videos for on-line and off-line recognition. The inter joint distances and the object are considered as low-level features for online classification. For off-line recognition, we propose rate-invariant classification of full video and early recognition. A shape analysis of trajectories of the inter-joint and object-joints distances is proposed for this end. The experiments conducted following state-of-the-art settings using MSR Daily Activity 3D Dataset and Online RGBD Action Dataset and on a new Multi-view dataset for human object interaction demonstrate that the proposed approach is effective and discriminative for human object interaction classification as demonstrated here.

show abstract

Informative joints based human action recognition using skeleton contexts

Cited by 67 publications

References 22 publications

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Distances evolution analysis for online and off-line human object interaction recognition

Contact Info

Product

Resources

About