PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation

Abstract

This paper addresses the challenge of 6DoF pose estimation from a single RGB image under severe occlusion or truncation. Many recent works have shown that a two-stage approach, which first detects keypoints and then solves a Perspective-n-Point (PnP) problem for pose estimation, achieves remarkable performance. However, most of these methods only localize a set of sparse keypoints by regressing their image coordinates or heatmaps, which are sensitive to occlusion and truncation. Instead, we introduce a Pixel-wise Voting Network (PVNet) to regress pixel-wise unit vectors pointing to the keypoints and use these vectors to vote for keypoint locations using RANSAC. This creates a flexible representation for localizing occluded or truncated keypoints. Another important feature of this representation is that it provides uncertainties of keypoint locations that can be further leveraged by the PnP solver. Experiments show that the proposed approach outperforms the state of the art on the LINEMOD, Occlusion LINEMOD and YCB-Video datasets by a large margin, while being efficient for real-time pose estimation. We further create a Truncation LINEMOD dataset to validate the robustness of our approach against truncation. The code will be avaliable at

Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement

“…Under the KPEC setting, we developed a monocular pose estimation technique for space-borne objects such as satellites. Inspired by works that combine the strength of deep neural networks and geometric optimisation [26,25,35], our approach contains three main components:…”

Multi-View Shape Estimation of Transparent Containers

“…Existing methods for localising objects in 3D or estimating their 6 DoF pose rely on databases of 3D object models [5,6,7,8] or need motion capture systems with markers [4,9,10]. To avoid using markers, feature points [11,12] can be localised in an image and matched against a 3D object model to estimate the object pose by solving a Perspective-n-Point (PnP) problem [13].…”

“…For example, DenseFusion [5] combines features obtained from RGB-D images and can handle occlusions and inaccurate segmentation. Pixel-wise Voting Network (PVNet) [6] estimates the pose of occluded or truncated objects with an uncertainty-driven PnP, learning a vector-field representation to localise a sparse set of 2D keypoints and their spatial uncertainty. Normalized Object Coordinate Space (NOCS) [16] uses a normalised object coordinates space formulation that jointly estimates the 6 DoF pose and the dimensions (in the form of a 3D bounding box) of a novel object (i.e.…”

“…an object whose shape was not seen during training). Table 1 compares relevant works, and comprehensive reviews of object pose estimation can be found in [5,6,16,17]. Although DNN models estimate the 6 DoF object pose quite accurately, their training requires large amount of data usually annotated only for the high-level object category, containing images and/or known dense 3D models [5,6,7,16,17].…”

“…Table 1 compares relevant works, and comprehensive reviews of object pose estimation can be found in [5,6,16,17]. Although DNN models estimate the 6 DoF object pose quite accurately, their training requires large amount of data usually annotated only for the high-level object category, containing images and/or known dense 3D models [5,6,7,16,17]. For example, PoseCNN [18], DenseFusion [5], SegOPE [17] and PVNet [6] evaluate only on objects with high-quality 3D models and good visibility in depth [18].…”