SO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation With Semi-Supervised Learning

Chen, Yujin; Tu, Zhigang; Ge, Liuhao; Zhang, Dejun; Chen, Ruizhi; Yuan, Junsong

doi:10.1109/iccv.2019.00706

Cited by 90 publications

(41 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Capsule-HandNet is compared with some of the state-of-the-art methods, including multi-view CNNs [6], LRF [3], the Deep Model [45], DeepPrior [40], DeepPrior++ [12], Crossing Nets [50], HBE [51], 3D CNN [7], V2V PoseNet [10], So-HandNet [9], LSN [52], Hierarchical [4], REN [53]. The fraction of frames and the per-joint mean error distances of different methods in MSRA and ICVL datasets are presented in Figures 4 and 5, respectively.…”

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

“…The fraction of frames and the per-joint mean error distances of different methods in MSRA and ICVL datasets are presented in Figures 4 and 5, respectively. The results of some methods are obtained from trained models available online [3,9,12,40,52,53] and others are cited from corresponding papers [4,6,7,10,45,50,51].…”

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

“…Thus, hand pose estimation from depth images has drawn growing research interest in recent years. With the development of deep neural networks in the field of computer vision and the emergence of large hand pose datasets [3][4][5], many 3D hand pose estimation methods have been applied and improved based on Convolutional Neural Networks (CNNs) [6][7][8][9][10][11][12][13][14]. A class of methods [6,14] project depth images onto multi-views and apply multi-view CNNs to regress the heat maps of these multi-views.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, these 3D CNN-based methods not only waste the calculation of 3D convolution, but also distract the neural network from learning effective features. In this situation, many studies on point cloud processing have started to focus on consuming point clouds directly to tackle the spatial information loss problems [8,9].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

3D Capsule Hand Pose Estimation Network Based on Structural Relationship Information

Zhang

et al. 2020

Symmetry

Self Cite

View full text Add to dashboard Cite

Hand pose estimation from 3D data is a key challenge in computer vision as well as an essential step for human–computer interaction. A lot of deep learning-based hand pose estimation methods have made significant progress but give less consideration to the inner interactions of input data, especially when consuming hand point clouds. Therefore, this paper proposes an end-to-end capsule-based hand pose estimation network (Capsule-HandNet), which processes hand point clouds directly with the consideration of structural relationships among local parts, including symmetry, junction, relative location, etc. Firstly, an encoder is adopted in Capsule-HandNet to extract multi-level features into the latent capsule by dynamic routing. The latent capsule represents the structural relationship information of the hand point cloud explicitly. Then, a decoder recovers a point cloud to fit the input hand point cloud via a latent capsule. This auto-encoder procedure is designed to ensure the effectiveness of the latent capsule. Finally, the hand pose is regressed from the combined feature, which consists of the global feature and the latent capsule. The Capsule-HandNet is evaluated on public hand pose datasets under the metrics of the mean error and the fraction of frames. The mean joint errors of Capsule-HandNet on MSRA and ICVL datasets reach 8.85 mm and 7.49 mm, respectively, and Capsule-HandNet outperforms the state-of-the-art methods on most thresholds under the fraction of frames metric. The experimental results demonstrate the effectiveness of Capsule-HandNet for 3D hand pose estimation.

show abstract

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

Section: Comparisons With State-of-the-art Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

3D Capsule Hand Pose Estimation Network Based on Structural Relationship Information

Zhang

et al. 2020

Symmetry

Self Cite

View full text Add to dashboard Cite

show abstract

“…Chen et al [107], in order to utilize depth images without 3D annotation, proposed a hand pose encoder-decoder network, where the encoder hierarchically extracted a vector representation from the point cloud and the decoder reconstructed the point cloud from the extracted vector. The encoder architecture was based on the SO-Net [108], which built a self-organizing map (SOM) [109] and performed a hierarchical feature extraction of a point cloud and SOM nodes.…”

Section: D Representation Utilizationmentioning

confidence: 99%

A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods

et al. 2020

View full text Add to dashboard Cite

The field of 3D hand pose estimation has been gaining a lot of attention recently, due to its significance in several applications that require human-computer interaction (HCI). The utilization of technological advances, such as cost-efficient depth cameras coupled with the explosive progress of Deep Neural Networks (DNNs), has led to a significant boost in the development of robust markerless 3D hand pose estimation methods. Nonetheless, finger occlusions and rapid motions still pose significant challenges to the accuracy of such methods. In this survey, we provide a comprehensive study of the most representative deep learning-based methods in literature and propose a new taxonomy heavily based on the input data modality, being RGB, depth, or multimodal information. Finally, we demonstrate results on the most popular RGB and depth-based datasets and discuss potential research directions in this rapidly growing field.

show abstract

Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation

Huang

Tan

et al. 2020

Lecture Notes in Computer Science

101

View full text Add to dashboard Cite

SO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation With Semi-Supervised Learning

Cited by 90 publications

References 35 publications

3D Capsule Hand Pose Estimation Network Based on Structural Relationship Information

3D Capsule Hand Pose Estimation Network Based on Structural Relationship Information

A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods

Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation

Contact Info

Product

Resources

About