Point-To-Pose Voting Based Hand Pose Estimation Using Residual Permutation Equivariant Layer

Li, Shile; Lee, Dongheui

doi:10.1109/cvpr.2019.01220

Cited by 82 publications

(72 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They usually require a good initialization; otherwise they are susceptible to getting stuck in local minima. Discriminative methods learn a direct mapping from visual observations to hand poses [23,27,10,13,31,2]. Thanks to large-scale annotated datasets [31,29,23], deep learningbased discriminative methods have shown very strong performance in the hand pose estimation task.…”

Section: Related Workmentioning

confidence: 99%

“…Moon et al [11] use 3D voxels as input and regress the hand pose with a 3D CNN. More recent works [10,5] apply 3D point clouds as input and can estimate very accurate hand poses.…”

Section: Related Workmentioning

confidence: 99%

“…Hand pose estimation plays an important role in areas such as human activity analysis, human computer interaction, and robotics. Depth-based 3D hand pose estimation methods are now highly accurate [25,10,28] largely due to advancements from deep learning. Despite commodity depth sensors being more commonplace, high-quality depth maps can still only be captured indoors, thereby limiting the environments in which depth-based methods can be deployed.…”

Section: Introductionmentioning

confidence: 99%

“…As such, the 3D hand pose and associated point cloud must be rotated such that the viewing angle towards the hand aligns with the optical axis. As indicated in [10], this correction is necessary to remove the many-to-one observation-pose pairings. We follow the approach given in [10].…”

mentioning

confidence: 99%

“…As indicated in [10], this correction is necessary to remove the many-to-one observation-pose pairings. We follow the approach given in [10]. Detailed equations on view correction can be found in the supplementary material.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Aligning Latent Spaces for 3D Hand Pose Estimation

Yang

Lee

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Hand pose estimation from monocular RGB inputs is a highly challenging task. Many previous works for monocular settings only used RGB information for training despite the availability of corresponding data in other modalities such as depth maps. In this work, we propose to learn a joint latent representation that leverages other modalities as weak labels to improve RGB-based hand pose estimation. By design, our architecture is highly flexible in embedding various diverse modalities such as heat maps, depth maps and point clouds. In particular, we find that encoding and decoding the point cloud of the hand surface can improve the quality of the joint latent representations. Experiments show that with the aid of other modalities during training, our proposed method boosts the accuracy of RGB-based hand pose estimation systems and significantly outperforms state-of-the-art on two public benchmarks.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Moon et al [11] use 3D voxels as input and regress the hand pose with a 3D CNN. More recent works [10,5] apply 3D point clouds as input and can estimate very accurate hand poses.…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Aligning Latent Spaces for 3D Hand Pose Estimation

Yang

Lee

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

JGR-P2O: Joint Graph Reasoning Based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Image

Fang

Xing-yan

Liu

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

State-of-the-art single depth image-based 3D hand pose estimation methods are based on dense predictions, including voxel-to-voxel predictions, point-to-point regression, and pixel-wise estimations. Despite the good performance, those methods have a few issues in nature, such as the poor trade-off between accuracy and efficiency, and plain feature representation learning with local convolutions. In this paper, a novel pixel-wise prediction-based method is proposed to address the above issues. The key ideas are two-fold: a) explicitly modeling the dependencies among joints and the relations between the pixels and the joints for better local feature representation learning; b) unifying the dense pixel-wise offset predictions and direct joint regression for end-toend training. Specifically, we first propose a graph convolutional network (GCN) based joint graph reasoning module to model the complex dependencies among joints and augment the representation capability of each pixel. Then we densely estimate all pixels' offsets to joints in both image plane and depth space and calculate the joints' positions by a weighted average over all pixels' predictions, totally discarding the complex postprocessing operations. The proposed model is implemented with an efficient 2D fully convolutional network (FCN) backbone and has only about 1.4M parameters. Extensive experiments on multiple 3D hand pose estimation benchmarks demonstrate that the proposed method achieves new state-of-the-art accuracy while running very efficiently with around a speed of 110fps on a single NVIDIA 1080Ti GPU 4 . The code is available at https://github.com/fanglinpu/JGR-P2O.

show abstract

Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation Under Hand-Object Interaction

Armagan

Garcia-Hernando

Baek

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We study how well different types of approaches generalise in the task of 3D hand pose estimation under single hand scenarios and handobject interaction. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set. Unfortunately, since the space of hand poses is highly dimensional, it is inherently not feasible to cover the whole space densely, despite recent efforts in collecting large-scale training datasets. This sampling problem is even more severe when hands are interacting with objects and/or inputs are RGB rather than depth images, as RGB images also vary with lighting conditions and colors. To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. More exactly, HANDS'19 is designed (a) to evaluate the influence of both depth and color modalities on 3D hand pose estimation, under the presence or absence of objects; (b) to assess the generalisation abilities w.r.t. four main axes: shapes, articulations, viewpoints, and objects; (c) to explore the use of a synthetic hand models to fill the gaps of current datasets. Through the challenge, the overall accuracy has dramatically improved over the baseline, especially on extrapolation tasks, from 27mm to 13mm mean joint error. Our analyses highlight the impacts of: Data pre-processing, ensemble approaches, the use of a parametric 3D hand model (MANO), and different HPE methods/backbones.

show abstract

Point-To-Pose Voting Based Hand Pose Estimation Using Residual Permutation Equivariant Layer

Cited by 82 publications

References 33 publications

Aligning Latent Spaces for 3D Hand Pose Estimation

Aligning Latent Spaces for 3D Hand Pose Estimation

JGR-P2O: Joint Graph Reasoning Based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Image

Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation Under Hand-Object Interaction

Contact Info

Product

Resources

About