Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images

Liu, Yuan; Wen, Yilin; Peng, Sida; Cheng, Lin; Long, Xiaoxiao; Komura, Taku; Wang, Wenping

doi:10.1007/978-3-031-19824-3_18

Cited by 58 publications

(37 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several techniques have been explored to generalize better to unseen object pose estimation, such as generic 2D-3D correspondences [31], an energy-based strategy [58], keypoint matching [43], or template matching [15,18,28,40,59]. Despite significant progress, these methods either need an accurate 3D model of the target or they rely on multiple annotated reference images from different viewpoints.…”

Section: Generalizable Object Pose Estimationmentioning

confidence: 99%

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Nguyen

Hu²,

Xiao

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

ReferenceQuery Predicted pose Pose distribution Reference Query Predicted pose Pose distribution Figure 1. Given as input a single reference view of a novel object, our method predicts the relative 3D pose of a query view and its ambiguities. We visualize the predicted pose by rendering the object from this pose, but the 3D model is only used for visualization purposes, not as input to our method. Our method works by estimating a probability distribution over the space of 3D poses, visualized here on a sphere centered on the object. We use the canonical pose of the 3D model to visualize this distribution, but not as input to our method. From this distribution, we can also identify the pose ambiguities: For example, in the case of the bottle, any pose with the same pitch and roll is possible; in the case of the mug, a range of poses are possible as the handle is not visible in the query image. Our method is also robust to partial occlusions, as shown on the clock hidden in part by a rectangle in the query image.

show abstract

Section: Generalizable Object Pose Estimationmentioning

confidence: 99%

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Nguyen

Hu²,

Xiao

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…However, conditioning on specific object categories limits the generalization to objects from novel categories with strong object variations. Meanwhile, some approaches [13,14,15,16] investigate generalizable 6D pose estimation as a few-shot learning problem, i.e., predicting the 6D pose of novel and category-agnostic objects given a few labeled reference images with the known pose of the novel object to define the object canonical coordinates. Although achieving promising results, these methods so far only work well on non-occluded and object-centric images, i.e., without the interference of other objects.…”

Section: Introductionmentioning

confidence: 99%

“…This limits the generalization to real-world scenarios with multiple objects in cluttered and occluded scenes. Furthermore, additional object information is required such as object diameter [13], mesh model [16], object 2D bounding box [15] or ground-truth mask [14,17], which is not always available for novel object categories. Our method aims to enable a fully generalizable few-shot 6D object pose estimation (FSPE) model.…”

Section: Introductionmentioning

confidence: 99%

“…In summary, we identify three primary challenges that are not adequately addressed by the current state-of-the-art methods [13,14,15,16]: i) The category-agnostic 6D pose estimation in cluttered scenes with heavy occlusions is performing poorly. ii) The object-centric reference images from cluttered scenes are cropped by ground-truth segmentation or bounding box of the target object, which limits the generalization in real-world scenarios.…”

Section: Introductionmentioning

confidence: 99%

“…Incorporating geometric features from the extracted point cloud, a region proposal module crops the test image by localizing the target object. With the cropped test image and the reference images, we employ Gen6D [13] to first predict an initial pose using visual input, followed by a refinement module using the induced geometric features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

What Matters For Meta-Learning Vision Regression Tasks?

Gao

Ziesche

Vien

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Meta-learning is widely used in few-shot classification and function regression due to its ability to quickly adapt to unseen tasks. However, it has not yet been well explored on regression tasks with high dimensional inputs such as images. This paper makes two main contributions that help understand this barely explored area. First, we design two new types of cross-category level vision regression tasks, namely object discovery and pose estimation of unprecedented complexity in the meta-learning domain for computer vision. To this end, we (i) exhaustively evaluate common meta-learning techniques on these tasks, and (ii) quantitatively analyze the effect of various deep learning techniques commonly used in recent meta-learning algorithms in order to strengthen the generalization capability: data augmentation, domain randomization, task augmentation and meta-regularization. Finally, we (iii) provide some insights and practical recommendations for training metalearning algorithms on vision regression tasks. Second, we propose the addition of functional contrastive learning (FCL) over the task representations in Conditional Neural Processes (CNPs) and train in an end-to-end fashion. The experimental results show that the results of prior work are misleading as a consequence of a poor choice of the loss function as well as too small meta-training sets. Specifically, we find that CNPs outperform MAML on most tasks without fine-tuning. Furthermore, we observe that naive task augmentation without a tailored design results in underfitting.

show abstract

MSDA: Monocular Self-supervised Domain Adaptation for 6D Object Pose Estimation

Cai

Heikkilä

Rahtu

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper introduces GS-Pose, an end-to-end framework for locating and estimating the 6D pose of objects. GS-Pose begins with a set of posed RGB images of a previously unseen object and builds three distinct representations stored in a database. At inference, GS-Pose operates sequentially by locating the object in the input image, estimating its initial 6D pose using a retrieval approach, and refining the pose with a render-and-compare method. The key insight is the application of the appropriate object representation at each stage of the process. In particular, for the refinement step, we utilize 3D Gaussian splatting, a novel differentiable rendering technique that offers high rendering speed and relatively low optimization time. Off-the-shelf toolchains and commodity hardware, such as mobile phones, can be used to capture new objects to be added to the database. Extensive evaluations on the LINEMOD and OnePose-LowTexture datasets demonstrate excellent performance, establishing the new state-of-the-art. Project page: https://dingdingcai.github.io/gs-pose

show abstract

Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images

Cited by 58 publications

References 64 publications

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

What Matters For Meta-Learning Vision Regression Tasks?

MSDA: Monocular Self-supervised Domain Adaptation for 6D Object Pose Estimation

Contact Info

Product

Resources

About