Refining 6D Object Pose Predictions using Abstract Render-and-Compare

Periyasamy, Arul Selvam; Schwarz, Max; Behnke, Sven

doi:10.1109/humanoids43949.2019.9035024

Cited by 16 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process is repeated until the estimated pose update is negligibly small. Refinement-based methods are orthogonal to the direct and indirect methods and are often used in combination with these methods [13,14,18,22,27], i.e., direct or indirect methods produce initial pose estimate and the refinement-based methods are used to refine the initial pose estimate to predict the final accurate pose estimate.…”

Section: Pose Estimationmentioning

confidence: 99%

T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regression

Amini¹,

Periyasamy²,

Behnke³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

6D pose estimation is the task of predicting the translation and orientation of objects in a given input image, which is a crucial prerequisite for many robotics and augmented reality applications. Lately, the Transformer Network architecture, equipped with multi-head self-attention mechanism, is emerging to achieve state-of-the-art results in many computer vision tasks. DETR, a Transformer-based model, formulated object detection as a set prediction problem and achieved impressive results without standard components like region of interest pooling, non-maximal suppression, and bounding box proposals. In this work, we propose T6D-Direct, a real-time single-stage direct method with a transformer-based architecture built on DETR to perform 6D multi-object pose direct estimation. We evaluate the performance of our method on the YCB-Video dataset. Our method achieves the fastest inference time, and the pose estimation accuracy is comparable to stateof-the-art methods.

show abstract

Section: Pose Estimationmentioning

confidence: 99%

T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regression

Amini¹,

Periyasamy²,

Behnke³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These approaches can achieve remarkable pose estimation performance by utilizing the ground-truth pose annotations to provide fully-supervised training signals. More recently, [26] attempted to train a 6D object pose refiner without real 6D pose labels via an approximated differentiable renderer, however the utilization of the differentiable renderer was non-trivial, since they required an additional representation learning step where real 6D pose annotations were still needed.…”

Section: Related Workmentioning

confidence: 99%

PFRL: Pose-Free Reinforcement Learning for 6D Pose Estimation

Shao

Jiang

et al. 2021

Preprint

View full text Add to dashboard Cite

6D pose estimation from a single RGB image is a challenging and vital task in computer vision. The current mainstream deep model methods resort to 2D images annotated with real-world ground-truth 6D object poses, whose collection is fairly cumbersome and expensive, even unavailable in many cases. In this work, to get rid of the burden of 6D annotations, we formulate the 6D pose refinement as a Markov Decision Process and impose on the reinforcement learning approach with only 2D image annotations as weakly-supervised 6D pose information, via a delicate reward definition and a composite reinforced optimization method for efficient and effective policy training. Experiments on LINEMOD and T-LESS datasets demonstrate that our Pose-Free approach is able to achieve state-of-theart performance compared with the methods without using real-world ground-truth 6D pose labels.

show abstract

“…We note that the renderer is equipped with approximative differentiation, allowing backpropagation of image-space gradients to object pose gradients. This functionality has been described by Periyasamy et al [18] in detail.…”

Section: Renderingmentioning

confidence: 99%

Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics

Schwarz

Behnke

2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Training data is the key ingredient for deep learning approaches, but difficult to obtain for the specialized domains often encountered in robotics. We describe a synthesis pipeline capable of producing training data for cluttered scene perception tasks such as semantic segmentation, object detection, and correspondence or pose estimation. Our approach arranges object meshes in physically realistic, dense scenes using physics simulation. The arranged scenes are rendered using high-quality rasterization with randomized appearance and material parameters. Noise and other transformations introduced by the camera sensors are simulated. Our pipeline can be run online during training of a deep neural network, yielding applications in life-long learning and in iterative render-andcompare approaches. We demonstrate the usability by learning semantic segmentation on the challenging YCB-Video dataset without actually using any training frames, where our method achieves performance comparable to a conventionally trained model. Additionally, we show successful application in a realworld regrasping system.

show abstract

Refining 6D Object Pose Predictions using Abstract Render-and-Compare

Cited by 16 publications

References 23 publications

T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regression

T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regression

PFRL: Pose-Free Reinforcement Learning for 6D Pose Estimation

Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics

Contact Info

Product

Resources

About