https://zc-alexfan.github.io/digit Figure 1. When estimating the 3D pose of interacting hands, state-of-the-art methods struggle to disambiguate the appearance of the two hands and their parts. In this example, the baseline fails to differentiate the left and the right wrists (1.1), resulting in erroneous pose estimation (1.2). Our model, DIGIT, reduces the ambiguity by predicting and leveraging a probabilistic part segmentation volume (2.1) to produce reliable pose estimates even when the two hands are in direct contact and under significant occlusion (2.2, 2.3).

adrian.spurr,zicong.fan,otmar.hilliges,siyu.tang}@inf.ethz.ch https://korrawe.github.io/halo Figure 1. We introduce a novel neural implicit surface representation of human hands (HALO) that is fully driven by keypoint-based skeleton articulation. Taking 3D keypoints as input, a fully differentiable implicit occupancy representation produces high-fidelity reconstruction of the hand surface (top row). We show that HALO facilitates the conditional generation of articulated hands that grasp 3D objects in a realistic and physically plausible manner (bottom row).

We introduce TempCLR, a new time-coherent contrastive learning approach for the structured regression task of 3D hand reconstruction. Unlike previous time-contrastive methods for hand pose estimation, our framework considers temporal consistency in its augmentation scheme, and accounts for the differences of hand poses along the temporal direction. Our data-driven method leverages unlabelled videos and a standard CNN, without relying on synthetic data, pseudo-labels, or specialized architectures. Our approach improves the performance of fully-supervised hand reconstruction methods by 15.9% and 7.6% in PA-V2V on the HO-3D and FreiHAND datasets respectively, thus establishing new state-of-the-art performance. Finally, we demonstrate that our approach produces smoother hand reconstructions through time, and is more robust to heavy occlusions compared to the previous state-of-the-art which we show quantitatively and qualitatively. Our code and models will be available at https://eth-ait.github.io/tempclr.

No abstract