3D Hand Pose Estimation From Monocular RGB With Feature Interaction Module

Guo, Shaoxiang; Rigall, Eric; Ju, Yakun; Dong, Junyu

doi:10.1109/tcsvt.2022.3142787

Cited by 16 publications

(3 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our scale attention is proposed to fuse the similarity response map, while most of the previous approaches use attentional mechanisms to improve the representation of features. (2) The implementations are different from previous work [24]. We apply global average pooling to extract the spatial information from the feature map, and then integrate all channels together by convolution.…”

Section: Attentional Mechanismsmentioning

confidence: 99%

See 1 more Smart Citation

SiamFFN: Siamese Feature Fusion Network for Visual Tracking

et al. 2023

View full text Add to dashboard Cite

Siamese network-based trackers have developed rapidly in the field of visual object tracking recently. Many Siamese network-based trackers currently in use rely on result fusion to combine the classification result map and regression result map. However, these result maps are obtained from the multi-level feature map and are independent of each other. It is inappropriate and flawed to use result fusion. Additionally, classification module and regression module are independent of each other, which leads to feature misalignment. In this paper, we propose a feature-fusion approach that involves fusing similarity response maps using a novel scale attention mechanism and subsequently decoding the features. To reduce the feature misalignment and produce more precise tracking results, we suggest using Classification Supervised Regression Loss (CSRL), to train the model. Experiments conducted on three challenging benchmark datasets show that this method outperforms current models in terms of both performance and efficiency, running at 40 fps.

show abstract

Section: Attentional Mechanismsmentioning

confidence: 99%

“…Object tracking is a basic challenge that involves predicting the target state in the video based on the initial state. It has several uses, including visual surveillance [1], pose estimation [2], and autonomous vehicles [3]. Therefore, it is a very active research direction.…”

Section: Introductionmentioning

confidence: 99%

SiamFFN: Siamese Feature Fusion Network for Visual Tracking

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This type of method can be further divided into detection-based and regression-based methods. The detection-based method is to generate a heat map to get the final predicted node joints [ 11 , 12 ], while the regression-based method is to directly regress the node position of all hand joints. A deep network was proposed in [ 13 ], which used a segmentation network to locate the hand, and output the scoring map of each joint point by PoseNet.…”

Section: Related Workmentioning

confidence: 99%

A 3D Hand Attitude Estimation Method for Fixed Hand Posture Based on Dual-View RGB Images

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

This work provides a 3D hand attitude estimation approach for fixed hand posture based on a CNN and LightGBM for dual-view RGB images to facilitate the application of hand posture teleoperation. First, using dual-view cameras and an IMU sensor, we provide a simple method for building 3D hand posture datasets. This method can quickly acquire dual-view 2D hand image sets and automatically append the appropriate three-axis attitude angle labels. Then, combining ensemble learning, which has strong regression fitting capabilities, with deep learning, which has excellent automatic feature extraction capabilities, we present an integrated hand attitude CNN regression model. This model uses a Bayesian optimization based LightGBM in the ensemble learning algorithm to produce 3D hand attitude regression and two CNNs to extract dual-view hand image features. Finally, a mapping from dual-view 2D images to 3D hand attitude angles is established using a training approach for feature integration, and a comparative experiment is run on the test set. The results of the experiments demonstrate that the suggested method may successfully solve the hand self-occlusion issue and accomplish 3D hand attitude estimation using only two normal RGB cameras.

show abstract