Accurate, Robust, and Flexible Real-time Hand Tracking

Sharp, Toby; Keskin, Cem; Robertson, Duncan; Taylor, Jonathan; Shotton, Jamie; Kim, David; Rhemann, Christoph; Leichter, Ido; Vinnikov, Alon; Wei, Yichen; Freedman, Daniel Z.; Kohli, Pushmeet; Krupka, Eyal; Fitzgibbon, Andrew; Izadi, Shahram

doi:10.1145/2702123.2702179

Cited by 389 publications

(312 citation statements)

References 31 publications

Supporting

Mentioning

305

Contrasting

Unclassified

Order By: Relevance

“…Many prior works have achieved good performance by different methods [1,2,3,4,5,6,7,8 Among the discriminative methods that learn the mapping from the depth images to the hand pose configurations, Sun et al [17] Fig. 1.…”

Section: Introductionmentioning

confidence: 99%

Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation

Yuan

Kim

2016

Lecture Notes in Computer Science

156

158

View full text Add to dashboard Cite

Abstract. Discriminative methods often generate hand poses kinematically implausible, then generative methods are used to correct (or verify) these results in a hybrid method. Estimating 3D hand pose in a hierarchy, where the high-dimensional output space is decomposed into smaller ones, has been shown effective. Existing hierarchical methods mainly focus on the decomposition of the output space while the input space remains almost the same along the hierarchy. In this paper, a hybrid hand pose estimation method is proposed by applying the kinematic hierarchy strategy to the input space (as well as the output space) of the discriminative method by a spatial attention mechanism and to the optimization of the generative method by hierarchical Particle Swarm Optimization (PSO). The spatial attention mechanism integrates cascaded and hierarchical regression into a CNN framework by transforming both the input(and feature space) and the output space, which greatly reduces the viewpoint and articulation variations. Between the levels in the hierarchy, the hierarchical PSO forces the kinematic constraints to the results of the CNNs. The experimental results show that our method significantly outperforms four state-of-the-art methods and three baselines on three public benchmarks.

show abstract

Section: Introductionmentioning

confidence: 99%

Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation

Yuan

Kim

2016

Lecture Notes in Computer Science

156

158

View full text Add to dashboard Cite

show abstract

“…PENDAHULUAN Pergerakan tangan manusia itu cukup tangkas, dinamis dan kompleks mampu melakukan pergerakan yang sistematis misalnya dalam proses mengetik, bermain piano, bermain gitar dan mungkin digunakan untuk komunikasi bahasa isyarat [1]. Bagi beberapa orang yang mempunyai gangguan penglihatan, pendengaran dan tunawicara tangan merupakan anggota badan yang utama dalam melakukan berbagai kegiatan.…”

Section: Kata Kunci-hand Tracking Inverse Kinematics Dof (Degree Ofunclassified

Teknik Hand Tracking Menggunakan Metode Inverse Kinematics Pada Pembuatan Animasi 3D

Pamujianto¹,

Suyanto²,

Sofyan³

2018

JOINTECS

View full text Add to dashboard Cite

Tahap produksi animasi 3D semakin berkembang. Banyak teknik – teknik baru yang terapkan pada setiap proses produksinya. Belum banyak produksi animasi yang menggunakan Teknik baru, sehingga memungkinkan rumah produksi dapat melakukan Teknik tersebut sebagai salah satu cara meningkatkan kualitas produksi animasi di dalam negeri. Salah satu Teknik yang sering digunakan dalam memproduksi animasi 3D adalah motion tracking. Teknik hand tracking merupakan salah satu dari Teknik motion tracking yang lebih berfokus pada organ tangan. Motion Tracking merupakan sebuah proses dalam mendapatkan tracker dari marker sebuah footage yang telah dipasangkan marker. Marker yang dihasilkan dari proses motion tracking dapat disebut sebagai tracer. Tracer dapat dihubungkan dengan objek 3D yang telah lengkap modeling dan rigging. Sehingga dapat menghasilkan animasi dari pergerakan tracer yang terhunbung dengan bone modeling 3d. Menggunakan pendekatan metode inverse kinematics dilakukan untuk memperbaiki pada saat animasi yang dihasilkan lebih terkontrol pergerakannya dan dengan menambahkan DOF (Degree Of Freedom) pada setiap ruang gerak model yang telah terhubungkan tracer dengan bone (tulang) riggingnya menghindari gerakan animasi keluar dari jalurnya.

show abstract

“…Schroder et al [20] suggest optimizing in a reduced parameter space and Tagliasacchi et al [24] combine previous results, to show that ICP in combination with temporal, collision, kinematic and data-driven terms can be utilized to track with high robustness and accuracy. Following up on this, Sharp et al [21] enhance this approach utilizing a smooth model and Tkach et al [27] present a new hand model based on sphere meshes. A non-gradient, particle swarm optimization (PSO) approach has been suggested by Oikonomidis et al [17], minimizing "the discrepancy between the appearance and 3D structure of hypothesized instances of a hand model and actual hand observations".…”

Section: Related Workmentioning

confidence: 99%

“…In methods utilizing PSO [17,29,18,21], we have seen that an error metric on depth images is meaningful enough to intelligently sample, compare and prune candidate poses, however the gradient idea has not been exploited. Encouraged by such results, we show in two experiments that we can optimize for the hand pose and shape.…”

Section: Feasibility Of Learning Hand Pose and Shapementioning

confidence: 99%

“…Next to convolutional data-driven approaches, there have been several generative, model-driven ones that perform iterative optimization. For instance, [13,20,24,26,27] optimize for point cloud correspondences while [17,21,29,18] attempt to find a good pose, by iteratively rendering many synthetic depth images and comparing them to the input image. Such approaches usually perform better on unseen poses, as compared to data-driven ones, when applied to poses quite dissimilar from the ones in training datasets.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How to Refine 3D Hand Pose Estimation from Unlabelled Depth Data?

Dibra

Wolf²,

Öztireli

et al. 2017

2017 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Data-driven approaches for hand pose estimation from depth images usually require a substantial amount of labelled training data which is quite hard to obtain. In this work, we show how a simple convolutional neural network, pre-trained only on synthetic depth images generated from a single 3D hand model, can be trained to adapt to unlabelled depth images from a real user's hand. We validate our method on two existing and a new dataset that we capture, both quantitatively and qualitatively, demonstrating that we strongly compare to state-of-the-art methods. Additionally, this method can be seen as an extension to existing methods trained on limited datasets, which helps on boosting their performance on new ones.

show abstract

Accurate, Robust, and Flexible Real-time Hand Tracking

Cited by 389 publications

References 31 publications

Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation

Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation

Teknik Hand Tracking Menggunakan Metode Inverse Kinematics Pada Pembuatan Animasi 3D

How to Refine 3D Hand Pose Estimation from Unlabelled Depth Data?

Contact Info

Product

Resources

About