ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

Shan, Mo; Feng, Qiaojun; Jau, You-Yi; Atanasov, Nikolay

doi:10.1109/iccv48922.2021.00589

Cited by 7 publications

(7 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vid2CAD [24], for instance, projects a CAD model to the multi-view observations to compute the loss for pose update. Other methods [30,39] employ neural 3D shape representations such as DeepSDF [28] to represent unknown shapes. ELLIPSDF [30], for instance, adopts a bi-level object shape model consisting of an ellipse and an SDF for estimation from point clouds observed as RGB-D videos.…”

Section: Related Workmentioning

confidence: 99%

“…Commonly used representations such as point clouds [7,8,21,33,38] and voxels [5,32] are inefficient as they first need to be instantiated in 3D before their 2D depth images can be rendered. Recently, implicit neural representations (INRs) have been introduced as a continuous but compact shape representation [5,29,30]. Typical examples of INRs, such as DeepSDF [28] and Occupancy Networks [26], have been used in conjunction with optimization pipelines [12,29,30], but they still require substantial computational and memory costs for rendering.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, implicit neural representations (INRs) have been introduced as a continuous but compact shape representation [5,29,30]. Typical examples of INRs, such as DeepSDF [28] and Occupancy Networks [26], have been used in conjunction with optimization pipelines [12,29,30], but they still require substantial computational and memory costs for rendering. The proposed DeepDDF directly outputs depth images from its latent representation.…”

Section: Related Workmentioning

confidence: 99%

“…Real-world Data We evaluate the accuracy and efficiency of TransPoser on ScanNet dataset [11], which consists of RGB-D videos of real-world indoor scenes. Following EL-LIPSDF [30], we use the ground-truth object pose, shape, and mask as well as the camera poses. The ground-truth object poses and shapes are provided by Scan2CAD [4].…”

Section: Ablation Studies On Synthetic Datamentioning

confidence: 99%

“…These methods either assume that the objects found in the scene are known and their shapes can be pre-acquired (i.e., detection and mapping) or only reconstruct the visible parts of them. Recent works have tackled joint estimation of the complete object shape and its pose from the observed RGB-D image sequence [30,39]. These methods, however, rely on conventional non-linear optimization, which fundamentally makes sequential online estimation hard.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation

Yoshitake¹,

Nishimura²,

Nobuhara³

et al. 2023

Preprint

View full text Add to dashboard Cite

We propose a novel method for joint estimation of shape and pose of rigid objects from their sequentially observed RGB-D images. In sharp contrast to past approaches that rely on complex non-linear optimization, we propose to formulate it as a neural optimization that learns to efficiently estimate the shape and pose. We introduce Deep Directional Distance Function (DeepDDF), a neural network that directly outputs the depth image of an object given the camera viewpoint and viewing direction, for efficient error computation in 2D image space. We formulate the joint estimation itself as a Transformer which we refer to as Trans-Poser. We fully leverage the tokenization and multi-head attention to sequentially process the growing set of observations and to efficiently update the shape and pose with a learned momentum, respectively. Experimental results on synthetic and real data show that DeepDDF achieves high accuracy as a category-level object shape representation and TransPoser achieves state-of-the-art accuracy efficiently for joint shape and pose estimation.

show abstract