Category-Level 6D Object Pose and Size Estimation Using Self-supervised Deep Prior Deformation Networks

Lin, Jiehong; Wei, Zewei; Ding, Changxing; Jia, Kui

doi:10.1007/978-3-031-20077-9_2

Cited by 41 publications

(33 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, many methods [21,3,30,35,4,2,5] have been proposed for category-level pose estimation, which can be categorized into two groups: prior-free methods and priorbased methods. Prior-free methods [4,22,34,33] mainly focus on designing network structures to fit the training data better.…”

Section: Prior-based Methodsmentioning

confidence: 99%

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Liu¹,

Chen²,

Ye³

et al. 2023

Preprint

View full text Add to dashboard Cite

Category-level 6D pose estimation aims to predict the poses and sizes of unseen objects from a specific category. Thanks to prior deformation, which explicitly adapts a category-specific 3D prior (i.e., a 3D template) to a given object instance, prior-based methods attained great success and have become a major research stream. However, obtaining category-specific priors requires collecting a large amount of 3D models, which is labor-consuming and often not accessible in practice. This motivates us to investigate whether priors are necessary to make prior-based methods effective. Our empirical study shows that the 3D prior itself is not the credit to the high performance. The keypoint actually is the explicit deformation process, which aligns camera and world coordinates supervised by worldspace 3D models (also called canonical space). Inspired by these observation, we introduce a simple prior-free implicit space transformation network, namely IST-Net, to transform camera-space features to world-space counterparts and build correspondence between them in an implicit manner without relying on 3D priors. Besides, we design camera-and world-space enhancers to enrich the features with pose-sensitive information and geometrical constraints, respectively. Albeit simple, IST-Net becomes the first prior-free method that achieves state-of-the-art performance, with top inference speed on the REAL275 dataset. Our code and models will be publicly available.

show abstract

Section: Prior-based Methodsmentioning

confidence: 99%

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Liu¹,

Chen²,

Ye³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Various follow-up works modify SPD's network architecture and training scheme to achieve further improvements. For example, CR-Net [14] uses a recurrent architecture to iteratively deform the canonical point set, SGPA [15] uses a transformer architecture to more effectively adjust the canonical point set, and DPDN [16] employs consistency-based losses for additional selfsupervised learning.…”

Section: Related Workmentioning

confidence: 99%

“…A notable line of work focusses on selfsupervised approaches. Following the early work by Manhardt et al [25], most works [48,50,16] focus on self-supervised learning in a sim-to-real context. That is, they first train in fully supervised fashion on synthetic data, and subsequently fine-tune without or with limited annotations on real data.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

Bruns¹

2023

Preprint

View full text Add to dashboard Cite

Recently, various methods for 6D pose and shape estimation of objects at a per-category level have been proposed. This work provides an overview of the field in terms of methods, datasets, and evaluation protocols. First, an overview of existing works and their commonalities and differences is provided. Second, we take a critical look at the predominant evaluation protocol, including metrics and datasets. Based on the findings, we propose a new set of metrics, contribute new annotations for the Redwood dataset, and evaluate state-of-the-art methods in a fair comparison. The results indicate that existing methods do not generalize well to unconstrained orientations and are actually heavily biased towards objects being upright. We provide an easy-to-use evaluation toolbox with well-defined metrics, methods, and dataset interfaces, which allows evaluation and comparison with various state-of-the-art approaches (https://github.com/roym899/pose and shape evaluation).

show abstract

“…Neural Shape Representation An efficient and accurate object shape representation is essential for iterative optimization for shape estimation as it dictates the computational cost of a single iteration. Commonly used representations such as point clouds [7,8,21,33,38] and voxels [5,32] are inefficient as they first need to be instantiated in 3D before their 2D depth images can be rendered. Recently, implicit neural representations (INRs) have been introduced as a continuous but compact shape representation [5,29,30].…”

Section: Related Workmentioning

confidence: 99%

TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation

Yoshitake¹,

Nishimura²,

Nobuhara³

et al. 2023

Preprint

View full text Add to dashboard Cite

We propose a novel method for joint estimation of shape and pose of rigid objects from their sequentially observed RGB-D images. In sharp contrast to past approaches that rely on complex non-linear optimization, we propose to formulate it as a neural optimization that learns to efficiently estimate the shape and pose. We introduce Deep Directional Distance Function (DeepDDF), a neural network that directly outputs the depth image of an object given the camera viewpoint and viewing direction, for efficient error computation in 2D image space. We formulate the joint estimation itself as a Transformer which we refer to as Trans-Poser. We fully leverage the tokenization and multi-head attention to sequentially process the growing set of observations and to efficiently update the shape and pose with a learned momentum, respectively. Experimental results on synthetic and real data show that DeepDDF achieves high accuracy as a category-level object shape representation and TransPoser achieves state-of-the-art accuracy efficiently for joint shape and pose estimation.

show abstract

Category-Level 6D Object Pose and Size Estimation Using Self-supervised Deep Prior Deformation Networks

Cited by 41 publications

References 25 publications

Prior-free Category-level Pose Estimation with Implicit Space Transformation

Prior-free Category-level Pose Estimation with Implicit Space Transformation

RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluation

TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation

Contact Info

Product

Resources

About