Fusing Posture and Position Representations for Point Cloud-Based Hand Gesture Recognition

Bigalke, Alexander; Heinrich, Mattias P.

doi:10.1109/3dv53792.2021.00071

Cited by 8 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Graph Convolutional Networks (GCNs) . These have gained significant attention in recent years for recognizing skeleton-based hand gestures [ 35 , 36 , 37 , 38 , 39 ]. These methods treat the skeleton as a graph, with joints as nodes and their spatial connections as edges.…”

Section: Related Workmentioning

confidence: 99%

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Zhong,

del-Blanco,

Berjón

et al. 2023

Sensors

View full text Add to dashboard Cite

Automatic hand gesture recognition in video sequences has widespread applications, ranging from home automation to sign language interpretation and clinical operations. The primary challenge lies in achieving real-time recognition while managing temporal dependencies that can impact performance. Existing methods employ 3D convolutional or Transformer-based architectures with hand skeleton estimation, but both have limitations. To address these challenges, a hybrid approach that combines 3D Convolutional Neural Networks (3D-CNNs) and Transformers is proposed. The method involves using a 3D-CNN to compute high-level semantic skeleton embeddings, capturing local spatial and temporal characteristics of hand gestures. A Transformer network with a self-attention mechanism is then employed to efficiently capture long-range temporal dependencies in the skeleton sequence. Evaluation of the Briareo and Multimodal Hand Gesture datasets resulted in accuracy scores of 95.49% and 97.25%, respectively. Notably, this approach achieves real-time performance using a standard CPU, distinguishing it from methods that require specialized GPUs. The hybrid approach’s real-time efficiency and high accuracy demonstrate its superiority over existing state-of-the-art methods. In summary, the hybrid 3D-CNN and Transformer approach effectively addresses real-time recognition challenges and efficient handling of temporal dependencies, outperforming existing methods in both accuracy and speed.

show abstract

Section: Related Workmentioning

confidence: 99%

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Zhong,

del-Blanco,

Berjón

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Although they achieved good performance, the main drawback is the lack of flexibility and sub-optimal performance because of the fixed graph structure, which may be difficult to capture variance and dynamics across different actions. To overcome the challenges, more recently, researchers have worked to develop a dynamic hand-skeleton dynamic graph-based spatialtemporal model to recognize hand gestures [25], [30], [31].…”

Section: Introductionmentioning

confidence: 99%

Dynamic Hand Gesture Recognition Using Multi-Branch Attention Based Graph and General Deep Learning Model

2023

View full text Add to dashboard Cite

The dynamic hand skeleton data have become increasingly attractive to widely studied for the recognition of hand gestures that contain 3D coordinates of hand joints. Many researchers have been working to develop skeleton-based hand gesture recognition systems using various discriminative spatialtemporal attention features by calculating the dependencies between the joints. However, these methods may face difficulties in achieving high performance and owing generalizability due to their inefficient features. To overcome these challenges, we proposed a Multi-branch attention-based graph and a general deep-learning model to recognize hand gestures by extracting all possible types of skeleton-based features. We used two graph-based neural network channels in our multi-branch architectures and one general neural network channel. In graph-based neural network channels, one channel first uses the spatial attention module and then the temporal attention module to produce the spatial-temporal features. In contrast, we produced temporalspatial features in the second channel using the reverse sequence of the first branch. In the last branch, general deep learning-based features are extracted using a general deep neural network module. The final feature vector was constructed by concatenating the spatial-temporal, temporal-spatial, and general features and feeding them into the fully connected layer. We included position embedding and mask operation for both spatial and temporal attention modules to track the sequence of the node and reduce the computational complexity of the system. Our model achieved 94.12%, 92.00%, and 97.01% accuracy after evaluation with MSRA, DHG, and SHREC'17 benchmark datasets, respectively. The high-performance accuracy and low computational cost proved that the proposed method outperformed the existing state-of-the-art methods.

show abstract

Gesture Recognition on Video Data

Gunkin,

Makarov

2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

Fusing Posture and Position Representations for Point Cloud-Based Hand Gesture Recognition

Cited by 8 publications

References 32 publications

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Dynamic Hand Gesture Recognition Using Multi-Branch Attention Based Graph and General Deep Learning Model

Gesture Recognition on Video Data

Contact Info

Product

Resources

About