Learning Discriminative Representations for Skeleton Based Action Recognition

Zhou, Huanyu; Liu, Qingjie; Wang, Yunhong

doi:10.1109/cvpr52729.2023.01022

Cited by 63 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although a single skeleton data can well represent the motion information [29, 30] whether indoor or outdoor, its disadvantage of not being able to recognize the appearance information is also fatal. For example, a drinking action and a food‐eating action have essentially the same direction of skeleton movement, but very different behaviours.…”

Section: Related Workmentioning

confidence: 99%

3sG: Three‐stage guidance for indoor human action recognition

Nan,

Ye,

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

Inference using skeleton to steer RGB videos is applicable to fine‐grained activities in indoor human action recognition (IHAR). However, existing methods that explore only spatial alignment are prone to bias, resulting in limited performance. The authors propose a Three‐stage Guidance (3sG) framework, leveraging skeleton knowledge to promote RGB in three stages. First, a soft shading image is proposed for alleviating background noise in videos, allowing the network to directly focus more on the motion region. Second, the authors propose to extract RGB frames of interest to reduce the computational effort. Furthermore, to explore more fully the complementary information between skeletons and RGB, the skeleton is coupled to the frame representation in a different spatial–temporal sharing pattern. Third, the global skeleton and skeleton‐guided RGB features are fed into the shared classifiers, which approximate the logit distributions of the two to enhance the performance in RGB unimodal. Finally, a fusion strategy that utilizes two learnable parameters to adaptively integrate the skeleton with the RGB is proposed. 3sG outperforms the state‐of‐the‐art results on the Toyota Smarthome dataset while it is more efficient than similar methods on the NTU RGB+D dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

3sG: Three‐stage guidance for indoor human action recognition

Nan,

Ye,

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…The model consists of an IAE-GCN module to extract interactive spatial structures and an IAM-TCN to extract temporal interactive features. Zhou et al [41] introduce a discriminative feature refinement module with contrastive learning to alleviate the problem of ambiguous action recognition from human skeletons.…”

Section: A Gcn-based Human Interaction Recognitionmentioning

confidence: 99%

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

Phuong,

Tran,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Graph Convolutional Networks (GCNs) have gained widespread adoption in modeling human skeleton sequences for two-person interaction recognition. Most GCN-based models achieve state-of-the-art results by leveraging either intra-body or inter-body connections. However, using only intra-body relations may ignore important interactive features between two individuals, whereas relying on inter-body relations may weaken the specific motion dynamics of each skeleton. To address these shortcomings, we propose a Distinct Motion-Preserving GCN (DMP-GCN) that utilizes intra-body and inter-body graphs to extract interactive features from two human bodies while preserving the distinct motion characteristics of each skeleton. Specifically, two motion-specific streams are adopted to capture specific motion features of each human skeleton and an interactive stream is applied to model the interactive dynamics of two bodies. In addition, we introduce a new graph labeling strategy called Distance Variation Labeling which is a datadriven approach for defining the edge strength in the skeleton graph. Extensive experiments show that our proposed approaches outperform state-of-the-art methods on two large-scale human interaction datasets, NTU RGB+D (mutual) and NTU RGB+D 120 (mutual).

show abstract

SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition

Do,

Kim

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning Discriminative Representations for Skeleton Based Action Recognition

Cited by 63 publications

References 34 publications

3sG: Three‐stage guidance for indoor human action recognition

3sG: Three‐stage guidance for indoor human action recognition

Distinct Motion-Preserving Graph Convolutional Network for Two-Person Interaction Recognition

SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition

Contact Info

Product

Resources

About