Spatial-aware stacked regression network for real-time 3D hand pose estimation

Ren, Pengfei; Sun, Haifeng; Huang, Weiting; Hao, Jiachang; Cheng, Daixuan; Qi, Qi; Wang, Jingyu; Liao, Jianxin

doi:10.1016/j.neucom.2021.01.045

Cited by 27 publications

(27 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, due to the adoption of a complex ResNet-50-based backbone network, A2J requires a higher computational complexity than the proposed method as described in Table 4. As compared to SRN [14] which shows the best performance in both 2D and 3D input data-based methods, our method achieves comparable results on ICVL and MSRA datasets. For NYU dataset, our method is inferior to SRN on accuracy but shows better running efficiency.…”

Section: Comparision With State-of-the-art Methodsmentioning

confidence: 90%

“…We compared the proposed network on three public 3D hand pose datasets with the most recently proposed methods using 2D depth maps as an input, including DISCO [35], Deep-Prior [36], its improved version DeepPrior++ [29], Feedback [16], Multi-view CNNs [34], REN-4 × 6 × 6 [38], REN-9 × 6 × 6 [39], Pose-REN [40], Generalized [41], Global2Local [10], CrossingNets [37], HBE [12], Cross-InfoNet [9], A2J [15], and SRN [14] as well as methods using 3D inputs, including 3D CNN [3], SHPR-Net [6], 3D DenseNet [4], HandPointNet [7], Point-to-Point [8], and V2V-PoseNet [5]. The average 3D distance error per joint and percentage of success frames over different error thresholds are respectively shown in Table 3 and Fig.…”

Section: Comparision With State-of-the-art Methodsmentioning

confidence: 99%

“…The smooth 1 loss is more robust to outliers than 2 loss and it is experimentally proven that smooth 1 loss is also effective in 3D hand pose estimation tasks [6], [14], [15]. In the joint regression SubNet, we define the global joint regression loss L g along with the local joint regression loss L l as the constraint to enforce each finger branch to be specialized in predefined subsets of the hand joints as follows:…”

Section: Loss Functionsmentioning

confidence: 99%

“…1 depicts the inference time (ms) versus mean distance error (mm) graph of some state-of-the-art 3D hand pose estimation methods on the NYU [13] dataset. Note that, in terms of estimation accuracy, the proposed method is ranked in the 5th place behind SRN [14], V2V-PoseNet [5], A2J [15], and Point-to-Point [8] which have higher computational complexity than the proposed method. The proposed HCRNN operates approximately 80, seven, and three times faster than V2V-PoseNet [5], Point-to-Point [8], and A2J [15], respectively.…”

Section: Introductionmentioning

confidence: 98%

See 3 more Smart Citations

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

Yoo

Shin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

3D hand pose estimation from a single depth image plays an important role in computer vision and human-computer interaction. Although recent hand pose estimation methods using convolution neural network (CNN) have shown notable improvements in accuracy, most of them have a limitation that they rely on a complex network structure without fully exploiting the articulated structure of the hand. A hand, which is an articulated object, is composed of six local parts: the palm and five independent fingers. Each finger consists of sequential-joints that provide constrained motion, referred to as a kinematic chain. In this paper, we propose a hierarchically-structured convolutional recurrent neural network (HCRNN) with six branches that estimate the 3D position of the palm and five fingers independently. The palm position is predicted via fully-connected layers. Each sequential-joint, i.e. finger position, is obtained using a recurrent neural network (RNN) to capture the spatial dependencies between adjacent joints. Then the output features of the palm and finger branches are concatenated to estimate the global hand position. HCRNN directly takes the depth map as an input without a time-consuming data conversion, such as 3D voxels and point clouds. Experimental results on public datasets demonstrate that the proposed HCRNN not only outperforms most 2D CNN-based methods using the depth image as their inputs but also achieves competitive results with state-of-the-art 3D CNN-based methods with a highly efficient running speed of 285 fps on a single GPU.INDEX TERMS 3D hand pose estimation, recurrent neural network, hand articulations.

show abstract

Section: Comparision With State-of-the-art Methodsmentioning

confidence: 90%

Section: Comparision With State-of-the-art Methodsmentioning

confidence: 99%

Section: Loss Functionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

See 2 more Smart Citations

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

Yoo

Shin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The "force" will drive a 2D hand skeleton to "fold" into the 3D joint coordinates representing the hand pose. adopted to perform direct hand depth image processing [40,10,14,30,3]. However, 2D CNNs cannot fully take advantage of 3D spatial information of the depth image, which is essential for achieving high accuracy.…”

Section: Introductionmentioning

confidence: 99%

HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton

Cheng¹,

Park

2021

Preprint

View full text Add to dashboard Cite

With increasing applications of 3D hand pose estimation in various human-computer interaction applications, convolution neural networks (CNNs) based estimation models have been actively explored. However, the existing models require complex architectures or redundant computational resources to trade with the acceptable accuracy. To tackle this limitation, this paper proposes HandFoldingNet, an accurate and efficient hand pose estimator that regresses the hand joint locations from the normalized 3D hand point cloud input. The proposed model utilizes a folding-based decoder that folds a given 2D hand skeleton into the corresponding joint coordinates. For higher estimation accuracy, folding is guided by multi-scale features, which include both global and joint-wise local features. Experimental results show that the proposed model outperforms the existing methods on three hand pose benchmark datasets with the lowest model parameter requirement. Code is available at https://github.com/cwc1260/HandFold.

show abstract

HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimation

Cheng,

Kim,

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Spatial-aware stacked regression network for real-time 3D hand pose estimation

Cited by 27 publications

References 22 publications

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton

HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimation

Contact Info

Product

Resources

About