PoseFlow: A Deep Motion Representation for Understanding Human Behaviors in Videos

Zhang, Dingwen; Guo, Guangyu; Huang, Dong; Han, Junwei

doi:10.1109/cvpr.2018.00707

Cited by 37 publications

(27 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HMMR learns a temporal representation for human dynamics by incorporating large-scale 2D pseudo-ground truth poses for in-the-wild videos. It uses PoseFlow (Zhang et al 2018)and Alpha-Pose (Fang et al 2017) for multi-person 2D pose estimation and tracking as a pre-processing step. Each person crop is then given as input to the CNN for estimating the pose and shape, as well as the weak-perspective camera parameters.…”

Section: D Human Motion Estimationmentioning

confidence: 99%

Synthetic Humans for Action Recognition from Unseen Viewpoints

et al. 2021

View full text Add to dashboard Cite

Although synthetic training data has been shown to be beneficial for tasks such as human pose estimation, its use for RGB human action recognition is relatively unexplored. Our goal in this work is to answer the question whether synthetic humans can improve the performance of human action recognition, with a particular focus on generalization to unseen viewpoints. We make use of the recent advances in monocular 3D human body reconstruction from real action sequences to automatically render synthetic training videos for the action labels. We make the following contributions: (1) we investigate the extent of variations and augmentations that are beneficial to improving performance at new viewpoints. We consider changes in body shape and clothing for individuals, as well as more action relevant augmentations such as non-uniform frame sampling, and interpolating between the motion of individuals performing the same action; (2) We introduce a new data generation methodology, SURREACT, that allows training of spatio-temporal CNNs for action classification; (3) We substantially improve the state-of-the-art action recognition performance on the NTU RGB+D and UESTC standard human action multi-view benchmarks; Finally, (4) we extend the augmentation approach to in-the-wild videos from a subset of the Kinetics dataset to investigate the case when only one-shot training data is available, and demonstrate improvements in this case as well.

show abstract

Section: D Human Motion Estimationmentioning

confidence: 99%

Synthetic Humans for Action Recognition from Unseen Viewpoints

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Video pose estimation has attracted less attention compared with image-based pose estimation mainly because of the limited number of large-scale benchmarks in video domain. Existing research works focus on extracting temporal cues, such as optical flow [1]- [3], [20], to help refine framewise estimation results generated by large networks. Song et al [1] propose a deep spatio-temporal network, namely Thin-Slicing, which aligns joint heatmaps across frames based on dense optical flow computation.…”

Section: B Human Pose Estimation In Videosmentioning

confidence: 99%

“…1(a), LSTM units are employed to transfer temporal knowledge as hidden states. Besides, optical flow is also widely exploited [1]- [3] The associate editor coordinating the review of this manuscript and approving it for publication was Shuhan Shen.…”

Section: Introductionmentioning

confidence: 99%

PosePropagationNet: Towards Accurate and Efficient Pose Estimation in Videos

Liu

Chen

2020

IEEE Access

View full text Add to dashboard Cite

We rethink on the contradiction between accuracy and efficiency in the field of video pose estimation. Large networks are typically exploited in previous methods to pursue superior pose estimation results. However, those methods can hardly meet the low-latency requirement for real-time applications because of their computationally expensive nature. We present a novel architecture, PosePropagation-Net (PPN), to generate poses across video frames accurately and efficiently. Instead of extracting temporal cues or knowledge someways to enforce geometric consistency as most of the previous methods do, we explicitly propagate well-estimated pose from the preceding frame to the current frame by leveraging pose propagation mechanism, endowing lightweight networks with the capability of performing accurate pose estimation in videos. The experiments on two large-scale benchmarks for video pose estimation show that our method significantly outperforms previous state-of-the-art methods in both accuracy and efficiency. Compared with the previous best method, our two representative configurations, PPN-Stable and PPN-Swift, achieve 2.5× and 6× FLOPs reduction respectively, as well as significant accuracy improvement. INDEX TERMS Network efficiency, pose propagation mechanism, video pose estimation.

show abstract

“…The performance of human action recognition has an important influence on many other tasks like video understanding and video surveillance. Many works have been proposed with different input modalities, including RGB video [1], [2], [9], [10], optical flow [4], [11] and human 2D/3D skeletons [8], [12], [13] (the optical flow and human skeletons can be estimated directly from the RGB video). Comparing to RGB video and optical flow, skeleton data is computationally more efficient and is robust to the variations in clothing and illumination.…”

Section: Introductionmentioning

confidence: 99%

Context-Aware Cross-Attention for Skeleton-Based Human Action Recognition

et al. 2020

View full text Add to dashboard Cite

Skeleton-based human action recognition is becoming popular due to its computational efficiency and robustness. Since not all skeleton joints are informative for action recognition, attention mechanisms are adopted to extract informative joints and suppress the influence of irrelevant ones. However, existing attention frameworks usually ignore helpful scenario context information. In this paper, we propose a cross-attention module that consists of a self-attention branch and a cross-attention branch for skeleton-based action recognition. It helps to extract joints that are not only more informative but also highly correlated to the corresponding scenario context information. Moreover, the cross-attention module maintains input variables' size and can be flexibly incorporated into many existing frameworks without breaking their behaviors. To facilitate end-to-end training, we further develop a scenario context information extraction branch to extract context information from raw RGB video directly. We conduct comprehensive experiments on the NTU RGB+D and the Kinetics databases, and experimental results demonstrate the correctness and effectiveness of the proposed model.

show abstract

PoseFlow: A Deep Motion Representation for Understanding Human Behaviors in Videos

Cited by 37 publications

References 20 publications

Synthetic Humans for Action Recognition from Unseen Viewpoints

Synthetic Humans for Action Recognition from Unseen Viewpoints

PosePropagationNet: Towards Accurate and Efficient Pose Estimation in Videos

Context-Aware Cross-Attention for Skeleton-Based Human Action Recognition

Contact Info

Product

Resources

About