3D Pose from Motion for Cross-View Action Recognition via Non-linear Circulant Temporal Encoding

Gupta, Ankur; Martínez, Julieta; Little, James J.; Woodham, Robert J.

doi:10.1109/cvpr.2014.333

Cited by 103 publications

(103 citation statements)

References 26 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…Lee and Chen [30] first investigated inferring 3D joints from their corresponding 2D projections. Later approaches either exploited nearest neighbors to refine the results of pose inference [18,25] or extracted hand-crafted features [1,23,47] for later regression. Other methods created over-complete bases which are suitable for representing human poses as sparse combinations [2,4,44,62,77].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Zhao

Peng

Tian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

517

398

View full text Add to dashboard Cite

In this paper, we study the problem of learning Graph Convolutional Networks (GCNs) for regression. Current architectures of GCNs are limited to the small receptive field of convolution filters and shared transformation matrix for each node. To address these limitations, we propose Semantic Graph Convolutional Networks (SemGCN), a novel neural network architecture that operates on regression tasks with graph-structured data. SemGCN learns to capture semantic information such as local and global node relationships, which is not explicitly represented in the graph. These semantic relationships can be learned through end-to-end training from the ground truth without additional supervision or hand-crafted rules. We further investigate applying SemGCN to 3D human pose regression. Our formulation is intuitive and sufficient since both 2D and 3D human poses can be represented as a structured graph encoding the relationships between joints in the skeleton of a human body. We carry out comprehensive studies to validate our method. The results prove that SemGCN outperforms state of the art while using 90% fewer parameters. The code can be found at https://github.com/garyzhao/SemGCN .

show abstract

Section: Related Workmentioning

confidence: 99%

“…They are complementary to the others. Remaining works target at exploiting temporal information [11,18,21,57] for 3D pose regression. They are out of the scope of this paper, since we aim at handling the 2D pose from one single image.…”

Section: Related Workmentioning

confidence: 99%

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Zhao

Peng

Tian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

517

398

View full text Add to dashboard Cite

show abstract

“…4. Specifically, given K human joints with [175] Vector of Joints Conc Lowlv Hand Patsadu et al [176] Vector of Joints Conc Lowlv Hand Huang and Kitani [177] Cost Topology Stat Lowlv Hand Devanne et al [178] Motion Units Conc Manif Hand Wang et al [179] Motion Poselets BoW Body Dict Wei et al [180] Structural Prediction Conc Lowlv Hand Gupta et al [181] 3D Pose w/o Body Parts Conc Lowlv Hand Amor et al [182] Skeleton's Shape Conc Manif Hand Sheikh et al [183] Action Space Conc Lowlv Hand Yilma and Shah [184] Multiview Geometry Conc Lowlv Hand Gong et al [185] Structured Time Conc Manif Hand Rahmani and Mian [186] Knowledge Transfer BoW Lowlv Dict Munsell et al [187] Motion Biometrics Stat Lowlv Hand Lillo et al [188] Composable Activities BoW Lowlv Dict Wu et al [189] Watch-n-Patch BoW Lowlv Dict Gong and Medioni [190] Dynamic Manifolds BoW Manif Dict Han et al [191] Hierarchical Manifolds BoW Manif Dict Slama et al [192,193] Grassmann Manifolds BoW Manif Dict Devanne et al [194] Riemannian Manifolds Conc Manif Hand Huang et al [195] Shape Tracking Conc Lowlv Hand Devanne et al [196] Riemannian Manifolds Conc Manif Hand Zhu et al [197] RNN with LSTM Conc Lowlv Deep Chen et al [198] EnwMi Learning BoW Lowlv Dict Hussein et al [199] Covariance of 3D Joints Stat Lowlv Hand Shahroudy et al [200] MMMP BoW Body Unsup Jung and Hong [201] Elementary Moving Pose BoW Lowlv Dict Evangelidis et al [202] Skeletal Quad Conc Lowlv Hand Azary and Savakis [203] Grassmann Manifolds Conc Manif Hand Barnachon et al [204] Hist. of Action Poses Stat Lowlv Hand Shahroudy et al [205] Feature Fusion BoW Body Unsup Cavazza et al [206] Kernelized-COV Stat Lowlv Hand …”

Section: Representations Based On Raw Joint Positionsmentioning

confidence: 99%

“…5. Gupta et al [181] proposed a cross-view human representation, which matches trajectory features of videos to MoCap joint trajectories and uses these matches to generate multiple motion projections as features. Junejo et al [207] used trajectorybased self-similarity matrices (SSMs) to encode humans observed from different views.…”

Section: Representations Based On Raw Joint Positionsmentioning

confidence: 99%

Space-time representation of people based on 3D skeletal data: A review

Han

Reily

Hoff

et al. 2017

Computer Vision and Image Understanding

276

192

View full text Add to dashboard Cite

Spatiotemporal human representation based on 3D visual perception data is a rapidly growing research area. Representations can be broadly categorized into two groups, depending on whether they use RGB-D information or 3D skeleton data. Recently, skeletonbased human representations have been intensively studied and kept attracting an increasing attention, due to their robustness to variations of viewpoint, human body scale and motion speed as well as the realtime, online performance. This paper presents a comprehensive survey of existing space-time representations of people based on 3D skeletal data, and provides an informative categorization and analysis of these methods from the perspectives, including information modality, representation encoding, structure and transition, and feature engineering. We also provide a brief overview of skeleton acquisition devices and construction methods, enlist a number of benchmark datasets with skeleton data, and discuss potential future research directions.

show abstract

“…Generating training data We adapt the mocap trajectory generation pipeline of Gupta et al [1], which uses a human model with cylindrical primitives (see Figure 1(b)). Each limb consists of a collection of points that are placed on a 3D surface.…”

Section: Methodsmentioning

confidence: 99%

Unlabelled 3D Motion Examples Improve Cross-View Action Recognition

Gupta

Shafaei

Little

et al. 2014

Proceedings of the British Machine Vision Conference 2014

Self Cite

View full text Add to dashboard Cite

(a) (b) Figure 1: (a) We exploit the visual similarity between mocap-generated trajectories (left) and dense trajectories (right) to improve cross-view action recognition. (b) For mocap-trajectories, we can easily obtain corresponding features (i.e., descriptors for trajectories that originate from the same 3D point) in two views. We use these pairs of features to learn the transformation function for viewpoint change. OverviewA view-invariant representation of human motion is crucial for effective action recognition. However, most view-invariant representations require either tracking of body parts or multi-view video data for learning which may not be a practical approach in many real-life scenarios. We describe a view-independent model for human action which is flexible, actionindependent, and requires no multi-view video data or additional labelling effort.We present a novel method for cross-view action recognition. Using a large collection of motion capture data we synthesize mocap-trajectory features from multiple viewpoints. Features originating from the same 3D point on the surface correspond, and this allows us to learn a feature transformation function for viewpoint change. Given this function, we can "hallucinate" the action descriptors of a video for different viewing angles. We use these hallucinated examples as additional training data to make our model view-invariant. We demonstrate the effectiveness of our approach on the unsupervised scenario of the INRIA IXMAS dataset. MethodologyThe approach has three steps:Generating training data We adapt the mocap trajectory generation pipeline of Gupta et al.[1], which uses a human model with cylindrical primitives (see Figure 1(b)). Each limb consists of a collection of points that are placed on a 3D surface. Given a camera viewpoint, these points are projected under orthographic projection and tracked for L(=15) consecutive frames to generate trajectory descriptors similar to the densetrajectories of Wang et al. [3]. The resulting displacement vectors are then used to generate trajectory features. Given two arbitrary viewpoints, we can find a correspondence between features that originate from the same point on the surface (see Figure 1(b)).Learning the transformation function We quantize the mocap trajectory features using a fixed codebook C of size n. Given a source camera elevation angle θ and relative change in viewpoint given by ∆ = (δ θ , δ φ ), we define the training set D ∆ θ = {( f i , g i )} m 1 to be the set of m pairs ( f , g) ∈ MethodAverage accuracy Ours 71.7% nCTE based matching [1] 67.4% w/o aug.62.1% Hankelets [2] 56.4%

show abstract

3D Pose from Motion for Cross-View Action Recognition via Non-linear Circulant Temporal Encoding

Cited by 103 publications

References 26 publications

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Semantic Graph Convolutional Networks for 3D Human Pose Regression

Space-time representation of people based on 3D skeletal data: A review

Unlabelled 3D Motion Examples Improve Cross-View Action Recognition

Contact Info

Product

Resources

About