VeREFINE: Integrating Object Pose Verification With Physics-Guided Iterative Refinement

Bauer, Dominik; Patten, Timothy; Vincze, Markus

doi:10.1109/lra.2020.2996059

Cited by 10 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the continuous improvement of AI and deep learning methods, it is expected that better object detection and pose estimation methods will appear. Since these methods do not degrade gracefully, we introduced a method for hypotheses verification [23], which uses physics simulation to verify if a pose hypothesis makes sense given the present data and structure of the environment. This also works towards explaining scenes at semantic level.…”

Section: Resultsmentioning

confidence: 99%

“…In our recent work [23] we presented the use of scene models in a feedback cycle to verify the plausibility of hypotheses. The scene model consists of physically annotated objects and structural elements, allowing our approach to inherently consider these properties in a rendering-based verification step using physics simulation.…”

Section: Fig 6 Accurate Pose Estimation Using Hypotheses Verificatimentioning

confidence: 99%

“…Success rate of pose estimates with less than 1 cm difference to ground truth, a standard measure in this field, is 91.9 % (up 2.9 percent over other works). For more details, please refer to [23].…”

Section: Fig 6 Accurate Pose Estimation Using Hypotheses Verificatimentioning

confidence: 99%

See 2 more Smart Citations

Learn, detect, and grasp objects in real-world settings

Vincze

Patten

Park

et al. 2020

Elektrotech. Inftech.

Self Cite

View full text Add to dashboard Cite

Experts predict that future robot applications will require safe and predictable operation: robots will need to be able to explain what they are doing to be trusted. To reach this goal, they will need to perceive their environment and its object to better understand the world and the tasks they have to perform. This article gives an overview of present advances with the focus on options to learn, detect, and grasp objects. With the approach of colour and depth (RGB-D) cameras and the advances in AI and deep learning methods, robot vision has been pushed considerably over the last years. We summarise recent results for pose estimation of objects and work on verifying object poses using a digital twin and physics simulation. The idea is that any hypothesis from an object detector and pose estimator is verified leveraging on the continuous advances in deep learning approaches to create object hypotheses. We then show that the object poses are robust enough such that a mobile manipulator can approach the object and grasp it. We intend to indicate that it is now feasible to model, recognise and grasp many objects with good performance, though further work is needed for applications in industrial settings.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Fig 6 Accurate Pose Estimation Using Hypotheses Verificatimentioning

confidence: 99%

See 1 more Smart Citation

Learn, detect, and grasp objects in real-world settings

Vincze

Patten

Park

et al. 2020

Elektrotech. Inftech.

Self Cite

View full text Add to dashboard Cite

show abstract

“…To improve performance in object pose estimation, a semantic segmentation head as proposed in the original PointNet paper [16] may be adapted to iteratively improve object segmentation during refinement. In addition, the combination of ReAgent with rendering-based verification such as proposed in [2] should be explored to efficiently consider multiple initial pose estimates. Further applications of the proposed method, such as self localization in large maps, will require efficient means to determine a state embedding from vast numbers of points.…”

Section: Discussionmentioning

confidence: 99%

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

Bauer¹,

Patten²,

Vincze³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Point cloud registration is a common step in many 3D computer vision tasks such as object pose estimation, where a 3D model is aligned to an observation. Classical registration methods generalize well to novel domains but fail when given a noisy observation or a bad initialization. Learning-based methods, in contrast, are more robust but lack in generalization capacity. We propose to consider iterative point cloud registration as a reinforcement learning task and, to this end, present a novel registration agent (ReAgent). We employ imitation learning to initialize its discrete registration policy based on a steady expert policy. Integration with policy optimization, based on our proposed alignment reward, further improves the agent's registration performance. We compare our approach to classical and learning-based registration methods on both ModelNet40 (synthetic) and ScanObjectNN (real data) and show that our ReAgent achieves state-of-the-art accuracy. The lightweight architecture of the agent, moreover, enables reduced inference time as compared to related approaches. In addition, we apply our method to the object pose estimation task on real data (LINEMOD), outperforming state-of-the-art pose refinement approaches. Code is available at github.com/dornik/reagent.

show abstract

“…T HE ability to visually identify objects in the scene and estimate their pose is fundamental for autonomous robotic systems. Several approaches have been proposed to tackle the problems of 6D object pose estimation [1], [2], [3], refinement [4], [5] and tracking [6], [7].…”

Section: Introductionmentioning

confidence: 99%

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

Piga

Onyshchuk

Pasquale

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

6D object pose tracking has been extensively studied in the robotics and computer vision communities. The most promising solutions, leveraging on deep neural networks and/or filtering and optimization, exhibit notable performance on standard benchmarks. However, to our best knowledge, these have not been tested thoroughly against fast object motions. Tracking performance in this scenario degrades significantly, especially for methods that do not achieve real-time performance and introduce non negligible delays. In this work, we introduce ROFT, a Kalman filtering approach for 6D object pose and velocity tracking from a stream of RGB-D images. By leveraging real-time optical flow, ROFT synchronizes delayed outputs of low frame rate Convolutional Neural Networks for instance segmentation and 6D object pose estimation with the RGB-D input stream to achieve fast and precise 6D object pose and velocity tracking. We test our method on a newly introduced photorealistic dataset, Fast-YCB, which comprises fast moving objects from the YCB model set, and on the dataset for object and hand pose estimation HO-3D. Results demonstrate that our approach outperforms state-of-the-art methods for 6D object pose tracking, while also providing 6D object velocity tracking. A video showing the experiments is provided as supplementary material.

show abstract

VeREFINE: Integrating Object Pose Verification With Physics-Guided Iterative Refinement

Cited by 10 publications

References 34 publications

Learn, detect, and grasp objects in real-world settings

Learn, detect, and grasp objects in real-world settings

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

Contact Info

Product

Resources

About