Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation

Kehl, Wadim; Milletarì, Fausto; Tombari, Federico; Ilić, Slobodan; Navab, Nassir

doi:10.1007/978-3-319-46487-9_13

Cited by 265 publications

(228 citation statements)

References 31 publications

Supporting

Mentioning

228

Contrasting

Order By: Relevance

“…The last method is to predict 3D locations of pixels or local shapes in the object space [2,16,23]. Brachmann et al [2] regress 3D coordinates and predict a class for each pixel using the auto-context random forest.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Park

Patten

Vincze

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

450

310

View full text Add to dashboard Cite

Estimating the 6D pose of objects using only RGB images remains challenging because of problems such as occlusion and symmetries. It is also difficult to construct 3D models with precise texture without expert knowledge or specialized scanning devices. To address these problems, we propose a novel pose estimation method, Pix2Pose, that predicts the 3D coordinates of each object pixel without textured models. An auto-encoder architecture is designed to estimate the 3D coordinates and expected errors per pixel. These pixel-wise predictions are then used in multiple stages to form 2D-3D correspondences to directly compute poses with the PnP algorithm with RANSAC iterations. Our method is robust to occlusion by leveraging recent achievements in generative adversarial training to precisely recover occluded parts. Furthermore, a novel loss function, the transformer loss, is proposed to handle symmetric objects by guiding predictions to the closest symmetric pose. Evaluations on three different benchmark datasets containing symmetric and occluded objects show our method outperforms the state of the art using only RGB images.

show abstract

Section: Related Workmentioning

confidence: 99%

“…A.1. Data augmentation for training2,3,4,13,14,15,16,17,18,24,30: sym = [I], the z-component of the rotation matrix is ignored. • Objects not in the list (non-symmetric): sym = [I] A.3.…”

mentioning

confidence: 99%

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Park

Patten

Vincze

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

450

310

View full text Add to dashboard Cite

show abstract

“…We first analyse the performance of the methods [36], [4], [2], [37] on the LINEMOD dataset. On the average, Kehl et al [36] outperforms other methods proving the superiority of learning deep features. Despite estimating 6D in RGB images, SSD-6D [37] exhibits the advantages of using CNN structures for 6D object pose estimation.…”

Section: A Analyses Based On Average Distancementioning

confidence: 99%

“…The studies in [28], [29] cope with texture-less objects. More recently, feature representations are learnt in an unsupervised fashion using deep convolutional (conv) networks (net) [35], [36]. While these methods fuse data coming from RGB and depth channels, a local belief propagation based approach [73] and an iterative refinement architecture [31], [32] are proposed in depth modality [74].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

Şahin

Garcia-Hernando

Sock

et al. 2020

Image and Vision Computing

View full text Add to dashboard Cite

Object pose recovery has gained increasing attention in the computer vision field as it has become an important problem in rapidly evolving technological areas related to autonomous driving, robotics, and augmented reality. Existing review-related studies have addressed the problem at visual level in 2D, going through the methods which produce 2D bounding boxes of objects of interest in RGB images. The 2D search space is enlarged either using the geometry information available in the 3D space along with RGB (Mono/Stereo) images, or utilizing depth data from LIDAR sensors and/or RGB-D cameras. 3D bounding box detectors, producing category-level amodal 3D bounding boxes, are evaluated on gravity aligned images, while full 6D object pose estimators are mostly tested at instance-level on the images where the alignment constraint is removed. Recently, 6D object pose estimation is tackled at the level of categories. In this paper, we present the first comprehensive and most recent review of the methods on object pose recovery, from 3D bounding box detectors to full 6D pose estimators. The methods mathematically model the problem as a classification, regression, classification & regression, template matching, and point-pair feature matching task. Based on this, a mathematical-model-based categorization of the methods is established. Datasets used for evaluating the methods are investigated with respect to the challenges, and evaluation metrics are studied. Quantitative results of experiments in the literature are analysed to show which category of methods best performs across what types of challenges. The analyses are further extended comparing two methods, which are our own implementations, so that the outcomes from the public results are further solidified. Current position of the field is summarized regarding object pose recovery, and possible research directions are identified.

show abstract

6D Pose Estimation of 3D Objects in Scenes with Mutual Similarities and Occlusions

Chen

Sun

2019

Proceedings of the International Conference on ISMAC in Computational Vision and Bio-Engineering 2018 (ISMAC-CVB)

View full text Add to dashboard Cite

Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation

Cited by 265 publications

References 31 publications

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

A review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators

6D Pose Estimation of 3D Objects in Scenes with Mutual Similarities and Occlusions

Contact Info

Product

Resources

About