Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks

Lee, Sang Hoon; Kim, Doyoung; Kang, Seoungyoon; Lee, Sanghoon

doi:10.1109/iccv.2017.115

Cited by 375 publications

(215 citation statements)

References 20 publications

Supporting

Mentioning

193

Contrasting

Order By: Relevance

“…In addition, since DD-net employs one-dimensional CNNs to extract the feature, it is much faster than other models that use RNNs [31], [22], [32], [25] or 2D/3D CNNs [5], [39], [7], [8], [28]. During its inferences, DD-Net's speed can reach around 3,500 FPS on one GPU (i.e., GTX 1080Ti), or, 2,000 FPS on one CPU (i.e., Intel E5-2620).…”

Section: Results Analysis and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Yang

Wu²,

Sakti

et al. 2019

Proceedings of the ACM Multimedia Asia

143

View full text Add to dashboard Cite

Although skeleton-based action recognition has achieved great success in recent years, most of the existing methods may suffer from a large model size and slow execution speed. To alleviate this issue, we analyze skeleton sequence properties to propose a Double-feature Double-motion Network (DD-Net) for skeleton-based action recognition. By using a lightweight network structure (i.e., 0.15 million parameters), DD-Net can reach a super fast speed, as 3,500 FPS on one GPU, or, 2,000 FPS on one CPU. By employing robust features, DD-Net achieves the state-of-the-art performance on our experiment datasets: SHREC (i.e., hand actions) and JHMDB (i.e., body actions). Our code is on https://github.com/fandulu/DD-Net.

show abstract

Section: Results Analysis and Discussionmentioning

confidence: 99%

“…1 (c)), solely using the JCD feature is insufficient. Unlike previous works that only utilize either the geometric feature [18], [22] or the Cartesian coordinate feature [24], [25], [26], [27], our DD-Net seamlessly integrates both of them.…”

Section: B Modeling Global Scale-invariant Motions By a Two-scale Momentioning

confidence: 99%

Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Yang

Wu²,

Sakti

et al. 2019

Proceedings of the ACM Multimedia Asia

143

View full text Add to dashboard Cite

show abstract

“…Note that there are essential differences between the proposed action reasoning approach and many deep learning based action recognition methods [8,9,23,36,44,45]: (1) Instead of only predicting a single action label, our method outputs multiple action labels with relevant objects, attributes/relationships and the time of each state transition. (2) Our action models are learned from semanticlevel state transitions based definitions (state detectors are trained on still images), and thus it does not need well-annotated video clips for training.…”

Section: Action Recognition Accuracymentioning

confidence: 99%

“…It is worth mentioning that TSN achieves the state-of-the-art performance 94.9% and 89.6% on the benchmark action recognition dataset UCF 101 [38] and ActivityNet [6], respectively. Similar to the other popular action recognition methods [8,9,23,36,44], the output of TSN is only an action label for a video sequence. In our experiment, the TSN model is trained with 50 epochs for both appearance and optical flow models, and the best recognition results are reported for comparison.…”

Section: Action Recognition Accuracymentioning

confidence: 99%

Explainable Video Action Reasoning via Prior Knowledge and State Transitions

Zhuo

Cheng

Zhang

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Human action analysis and understanding in videos is an important and challenging task. Although substantial progress has been made in past years, the explainability of existing methods is still limited. In this work, we propose a novel action reasoning framework that uses prior knowledge to explain semantic-level observations of video state changes. Our method takes advantage of both classical reasoning and modern deep learning approaches. Specifically, prior knowledge is defined as the information of a target video domain, including a set of objects, attributes and relationships in the target video domain, as well as relevant actions defined by the temporal attribute and relationship changes (i.e. state transitions). Given a video sequence, we first generate a scene graph on each frame to represent concerned objects, attributes and relationships. Then those scene graphs are linked by tracking objects across frames to form a spatio-temporal graph (also called video graph), which represents semantic-level video states. Finally, by sequentially examining each state transition in the video graph, our method can detect and explain how those actions are executed with prior knowledge, just like the logical manner of thinking by humans. Compared to previous works, the action reasoning results of our method can be explained by both logical rules and semantic-level observations of video content changes. Besides, the proposed method can be used to detect multiple concurrent actions with detailed information, such as who (particular objects), when (time), where (object locations) and how (what kind of changes). Experiments on a re-annotated dataset CAD-120 show the effectiveness of our method.

show abstract

“…For small human action recognition datasets, deep learning methods may not give the best performance. Recent Kinect-based human action recognition algorithms are: [14,18,[20][21][22][25][26][27][28][29][30][31][32][33][34][35][36][37][38]. Research contributions.…”

Section: Introductionmentioning

confidence: 99%

A Comparative Review of Recent Kinect-Based Action Recognition Algorithms

Wang

Huynh

Koniusz

2020

IEEE Trans. on Image Process.

218

View full text Add to dashboard Cite

Video-based human action recognition is currently one of the most active research areas in computer vision. Various research studies indicate that the performance of action recognition is highly dependent on the type of features being extracted and how the actions are represented. Since the release of the Kinect camera, a large number of Kinect-based human action recognition techniques have been proposed in the literature. However, there still does not exist a thorough comparison of these Kinect-based techniques under the grouping of feature types, such as handcrafted versus deep learning features and depthbased versus skeleton-based features. In this paper, we analyze and compare ten recent Kinect-based algorithms for both crosssubject action recognition and cross-view action recognition using six benchmark datasets. In addition, we have implemented and improved some of these techniques and included their variants in the comparison. Our experiments show that the majority of methods perform better on cross-subject action recognition than cross-view action recognition, that skeleton-based features are more robust for cross-view recognition than depth-based features, and that deep learning features are suitable for large datasets.

show abstract

Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks

Cited by 375 publications

References 20 publications

Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Explainable Video Action Reasoning via Prior Knowledge and State Transitions

A Comparative Review of Recent Kinect-Based Action Recognition Algorithms

Contact Info

Product

Resources

About