Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition

Liu, Ziyu; Zhang, Hongwen; Chen, Zhenghao; Wang, Zhiyong; Ouyang, Wanli

doi:10.1109/cvpr42600.2020.00022

Cited by 844 publications

(634 citation statements)

References 22 publications

Supporting

Mentioning

632

Contrasting

Order By: Relevance

“…Bone has proven to be another form of spatial information as important as joints and has been used in many recent methods [ 32 , 37 , 38 , 41 , 42 , 43 , 44 ]. The original skeleton data only contain the 3D coordinates of all joints in the skeleton, and the bone stream data are obtained by vector calculation of the original joint stream data.…”

Section: Methodsmentioning

confidence: 99%

“…All of our experiments were implemented on the PyTorch deep learning framework, and our model was trained on 4 GTX-1080Ti GPUs. In the data processing, we were consistent with [ 32 , 37 , 38 , 41 , 42 ] to ensure the fairness of model comparison. We explored two different training methods: one was the end-to-end training method, and the other was the two-stage training method.…”

Section: Methodsmentioning

confidence: 99%

“…Gao et al [ 40 ] designed BAGCN, which learns spatial-temporal context from human skeleton sequences leveraging a graph convolutional neural network-based focusing and diffusion mechanism. Liu et al [ 41 ] introduced a convolution operator of a cross-spatiotemporal aggregation graph to enhance the cross-spatiotemporal aggregation ability of skeleton information. These methods use graph convolution to extract the spatiotemporal features of the human skeleton.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Whole and Part Adaptive Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

Zuo

Zou

Fan

et al. 2020

Sensors

View full text Add to dashboard Cite

Spatiotemporal graph convolution has made significant progress in skeleton-based action recognition in recent years. Most of the existing graph convolution methods take all the joints of the human skeleton as the overall modeling graph, ignoring the differences in the movement patterns of various parts of the human, and cannot well connect the relationship between the different parts of the human skeleton. To capture the unique features of different parts of human skeleton data and the correlation of different parts, we propose two new graph convolution methods: the whole graph convolution network (WGCN) and the part graph convolution network (PGCN). WGCN learns the whole scale skeleton spatiotemporal features according to the movement patterns and physical structure of the human skeleton. PGCN divides the human skeleton graph into several subgraphs to learn the part scale spatiotemporal features. Moreover, we propose an adaptive fusion module that combines the two features for multiple complementary adaptive fusion to obtain more effective skeleton features. By coupling these proposals, we build a whole and part adaptive fusion graph convolution neural network (WPGCN) that outperforms previous state-of-the-art methods on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Whole and Part Adaptive Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

Zuo

Zou

Fan

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Recently, various types of methods have been proposed that have obtained good results for this problem. The best-performing solutions for methods based on 3D skeleton joints [ 30 , 31 , 32 , 33 ] combining the advantages of two important types of networks: Temporal Convolutional Neural Network [ 34 ] used to capture temporal dependencies and Graph Convolutional Neural Network [ 35 ] used to model spatial dependencies. In contrast, the solution we propose, starting from the analyzed models and the observations that we presented in the previous works [ 36 , 37 ], is based on a multi-stage architecture based on linear layers used to extract features and long short-term memory layers used to model the sequence of frames to obtain the correct action.…”

Section: Related Workmentioning

confidence: 99%

The AMIRO Social Robotics Framework: Deployment and Evaluation on the Pepper Robot

Ghita

Gavril

Nan

et al. 2020

Sensors

View full text Add to dashboard Cite

Recent studies in social robotics show that it can provide economic efficiency and growth in domains such as retail, entertainment, and active and assisted living (AAL). Recent work also highlights that users have the expectation of affordable social robotics platforms, providing focused and specific assistance in a robust manner. In this paper, we present the AMIRO social robotics framework, designed in a modular and robust way for assistive care scenarios. The framework includes robotic services for navigation, person detection and recognition, multi-lingual natural language interaction and dialogue management, as well as activity recognition and general behavior composition. We present AMIRO platform independent implementation based on a Robot Operating System (ROS). We focus on quantitative evaluations of each functionality module, providing discussions on their performance in different settings and the possible improvements. We showcase the deployment of the AMIRO framework on a popular social robotics platform—the Pepper robot—and present the experience of developing a complex user interaction scenario, employing all available functionality modules within AMIRO.

show abstract

“…While human arXiv:2012.09402v1 [cs.CV] 17 Dec 2020 perception typically involves inferring the physical attributes about the humans (detection [5,35,43,50], poses [3,4,8,25,28,41], shape [13,20,29,30], gaze [44] etc. ), interpreting humans involves reasoning about the finer details relating to human activity [6,24,27,48,49], behaviour [26,34], human-object visual relationship detection [23,33,36,37,39,40], and human-object interactions [23,32,33,36,37,39,40,42]. In this work, we investigate the problem of identifying Human-Object Interactions in videos.…”

Section: Introductionmentioning

confidence: 99%

Lighten

Sunkesula

Dabral

Ramakrishnan

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Analyzing the interactions between humans and objects from a video includes identification of the relationships between humans and the objects present in the video. It can be thought of as a specialized version of Visual Relationship Detection, wherein one of the objects must be a human. While traditional methods formulate the problem as inference on a sequence of video segments, we present a hierarchical approach, LIGHTEN, to learn visual features to effectively capture spatio-temporal cues at multiple granularities in a video. Unlike current approaches, LIGHTEN avoids using ground truth data like depth maps or 3D human pose, thus increasing generalization across non-RGBD datasets as well. Furthermore, we achieve the same using only the visual features, instead of the

show abstract

Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition

Cited by 844 publications

References 22 publications

Whole and Part Adaptive Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

Whole and Part Adaptive Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

The AMIRO Social Robotics Framework: Deployment and Evaluation on the Pepper Robot

Lighten

Contact Info

Product

Resources

About