Graph Diffusion Convolutional Network for Skeleton Based Semantic Recognition of Two-Person Actions

Li, Shuai; He, Xinxue; Song, Wenfeng; Hao, Aimin; Qin, Hong

doi:10.1109/tpami.2023.3238411

Cited by 8 publications

(2 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhu et al [33] present a Relational Adjacency Matrix that demonstrates dynamic relational graphs and supervises the model to learn spatial-temporal interactive features from two skeleton sequences. Li et al [47] design a diffusion graph structure based on practical action information to construct connections between joints from the same body and joints from two interacting people. Ito et al [34], [35] apply intrabody and inter-body graphs in multi-stream networks to ensure high performance for recognizing human interaction.…”

Section: B Graph Topology Construction Methodsmentioning

confidence: 99%

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

Phuong,

Tran,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Graph Convolutional Networks (GCNs) have gained widespread adoption in modeling human skeleton sequences for two-person interaction recognition. Most GCN-based models achieve state-of-the-art results by leveraging either intra-body or inter-body connections. However, using only intra-body relations may ignore important interactive features between two individuals, whereas relying on inter-body relations may weaken the specific motion dynamics of each skeleton. To address these shortcomings, we propose a Distinct Motion-Preserving GCN (DMP-GCN) that utilizes intra-body and inter-body graphs to extract interactive features from two human bodies while preserving the distinct motion characteristics of each skeleton. Specifically, two motion-specific streams are adopted to capture specific motion features of each human skeleton and an interactive stream is applied to model the interactive dynamics of two bodies. In addition, we introduce a new graph labeling strategy called Distance Variation Labeling which is a datadriven approach for defining the edge strength in the skeleton graph. Extensive experiments show that our proposed approaches outperform state-of-the-art methods on two large-scale human interaction datasets, NTU RGB+D (mutual) and NTU RGB+D 120 (mutual).

show abstract

Section: B Graph Topology Construction Methodsmentioning

confidence: 99%

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

Phuong,

Tran,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To fully utilize the topological graph structure of the human skeleton and capture spatial dependency between joints, GCNs [ 38 ] were introduced. Li et al [ 39 ] proposes a Graph Diffusion Convolutional Network(GDCN) approach that integrates graph diffusion and GCNs for enhanced twoperson action recognition. Yan et al [ 9 ] proposed the ST-GCN architectures to represent the skeleton sequence as a spatial-temporal graph, enabling comprehensive capture of human behavior's spatial-temporal change relationship and achieving unprecedented recognition accuracy.…”

Section: A Skeleton-based Group Activity Recognitionmentioning

confidence: 99%

Attention Relational Network for Skeleton-Based Group Activity Recognition

Wang,

Mohamed

2023

IEEE Access

View full text Add to dashboard Cite

Group Activity Recognition refers to multi-individual activity analysis. It is a significant and challenging task in computer vision. The solution of group activity prediction can be classified with traditional hand-crafted features, RGB video features, and skeleton data-based deep learning architectures, such as Graph Convolutional Networks (GCNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTMs). However, not only are those solutions usually designed to be complex, but they also rarely explore pose information and rarely use relational networks to reason about group activity behavior. They disregard the magnitude and orientation details of the skeletal edges, crucial for action recognition, potentially leading to suboptimal outcomes in these methodologies. In this work, we leverage minimal prior knowledge about the skeleton information to reason about the interactions from group activity. The objective is to obtain discriminative representations and filter out some ambiguous actions to enhance the performance of group activity recognition. Our contribution is a proposed Attention Relation Network (ARN) that fuses the attention mechanisms and joint vector sequences into the relation network. The skeleton joints vector sequences are previously unexplored pose information and assign greater significance attributed to individuals who are more relevant for distinguishing the group activity behavior. In the first stage, our model focuses on the specified edge-level information (encompassing both edge and edge motion data) within the skeleton dataset, considering directionality, to analyze the spatiotemporal aspects of the action. In the second stage, recognizing the inherent motion directionality, we establish diverse directions for skeleton edges and extract distinct motion features (including translation and rotation information) aligned with these various orientations, thereby augmenting the utilization of motion attributes related to the action. We also introduce a representation of human motion achieved by combining relational networks and examining their integrated characteristics. Extensive experiments were tested in the Hockey and UT-interaction datasets to evaluate our method, obtaining competitive performance to the state-of-the-art. Results demonstrate the modeling potential of a skeletonbased method for group activity recognition.

show abstract