Pairwise Adjacency Matrix on Spatial Temporal Graph Convolution Network for Skeleton-Based Two-Person Interaction Recognition

Yang, Chao-Lung; Setyoko, Aji; Tampubolon, Hendrik; Hua, Kai-Lung

doi:10.1109/icip40778.2020.9190680

Cited by 20 publications

(20 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• GCN-based methods: ST-GCN [23], AS-GCN [56], 2s-AGCN [24], ST-GCN-PAM [35], and MS-AGCN [27] Note that ST-GCN, 2s-AGCN, and MS-AGCN were implemented and evaluated ourselves. The scores for the other comparison methods were taken from [32], [35].…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…Several studies do not use skeleton information [29]- [31], [33], while others transform skeleton information (e.g., the coordinates of joints) into hand-crafted features [32], [34]. In addition, ST-GCN-PAM [35] has been proposed, which utilizes ST-GCN to capture the physical connection of the human body.…”

Section: A Human Interaction Recognitionmentioning

confidence: 99%

“…Therefore, we newly propose the use of the 2Bp-stream in conjunction with the above two streams. The core idea of the 2Bp-stream was introduced in [35]. This stream also utilizes the intra-body graph as a predefined graph to capture the inter-body relative distances and the direction between two people.…”

Section: ) Streams Using Intra-body Graphmentioning

confidence: 99%

“…Several interaction recognition methods have been proposed, with their approaches primarily based on an RNN or CNN model using RGB data [29]- [31], [33] or skeleton data [32], [34]. However, GCN-based methods such as ST-GCN pairwise adjacency matrix (ST-GCN-PAM) [35] have generally been more accurate than these methods. ST-GCN-PAM directly represents the relation between two people by calculating the distance from one person's joint to the other person's joint in the same position, e.g., "One person's head and the other person's head."…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-Stream Adaptive Graph Convolutional Network Using Inter- and Intra-Body Graphs for Two-Person Interaction Recognition

et al. 2021

View full text Add to dashboard Cite

Two-person interaction recognition has become an area of growing interest in human action recognition. The graph convolutional network (GCN) using human skeleton data has been shown to be highly effective for action recognition. Most GCN-based methods focus on recognizing an individual person's actions on the basis of an intra-body graph. However, many of these methods do not represent the relation between two bodies, making it difficult to accurately recognize human interaction. In this work, we propose multi-stream adaptive GCN using inter-and intra-body graphs (MAGCN-IIG) as a new method of human interaction recognition. To ensure highly accurate human interaction recognition, our method cooperatively utilizes two types of graphs: an inter-body graph and an intra-body graph. The inter-body graph, which is newly introduced in this paper, connects the inter-body joints between two people as well as intra-body connections. The adaptive GCN using the inter-body graph captures the relation of joints between two people, even different types of joints located far away from each other. Further, by implementing a multi-stream architecture, our method simultaneously captures both inter-body and intra-body relations in each of two units that represent the position and motion of people. Experiments on interaction recognition using two large-scale human action datasets, NTU RGB+D and NTU RGB+D 120, showed that our method recognized human interactions more accurately than state-of-the-art methods.INDEX TERMS Graph convolutional network, human interaction recognition, multi-stream network, skeleton-based action recognition.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: A Human Interaction Recognitionmentioning

confidence: 99%

Section: ) Streams Using Intra-body Graphmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Stream Adaptive Graph Convolutional Network Using Inter- and Intra-Body Graphs for Two-Person Interaction Recognition

et al. 2021

View full text Add to dashboard Cite

show abstract

“…There have been multiple action recognition systems using the ST-GCN architecture [5], [16]- [18]. Zheng et al [12] extracted the skeleton from the UCF-101 dataset in a similar manner as Yan et.…”

Section: A Partitioning Strategiesmentioning

confidence: 99%

Skeleton-Split Framework using Spatial Temporal Graph Convolutional Networks for Action Recogntion

Alsawadi¹,

Rio²

2021

Preprint

View full text Add to dashboard Cite

There has been a dramatic increase in the volume of videos and their related content uploaded to the internet. Accordingly, the need for efficient algorithms to analyse this vast amount of data has attracted significant research interest. An action recognition system based upon human body motions has been proven to interpret videos' contents accurately. This work aims to recognize activities of daily living using the ST-GCN model, providing a comparison between four different partitioning strategies: spatial configuration partitioning, full distance split, connection split, and index split. To achieve this aim, we present the first implementation of the ST-GCN framework upon the HMDB-51 dataset. We have achieved 48.88% top-1 accuracy by using the connection split partitioning approach. Through experimental simulation, we show that our proposals have achieved the highest accuracy performance on the UCF-101 dataset using the ST-GCN framework than the state-of-the-art approach. Finally, accuracy of 73.25% top-1 is achieved by using the index split partitioning strategy.

show abstract

Dynamic Skeleton Association Transformer for Dyadic Interaction Action Recognition

Liu,

Zhang,

Zhao

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Pairwise Adjacency Matrix on Spatial Temporal Graph Convolution Network for Skeleton-Based Two-Person Interaction Recognition

Cited by 20 publications

References 14 publications

Multi-Stream Adaptive Graph Convolutional Network Using Inter- and Intra-Body Graphs for Two-Person Interaction Recognition

Multi-Stream Adaptive Graph Convolutional Network Using Inter- and Intra-Body Graphs for Two-Person Interaction Recognition

Skeleton-Split Framework using Spatial Temporal Graph Convolutional Networks for Action Recogntion

Dynamic Skeleton Association Transformer for Dyadic Interaction Action Recognition

Contact Info

Product

Resources

About