CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

Weng, Yijia; Wang, He; Zhou, Qiang; Qin, Yuzhe; Duan, Yueqi; Fan, Qingnan; Chen, Baoquan; Su, Hao; Guibas, Leonidas J.

doi:10.48550/arxiv.2104.03437

Cited by 9 publications

(15 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The benefit of exploiting temporal consistency is obvious by comparing the single frame and tracking performance in Table I. We compare our category-level 6D object pose tracking system with ICP [46], 6-PACK [14], CAPTRA [15], and BundleTrack [16]. Overall, our system achieves better performance compared to 6-PACK and ICP in terms of overall IoU25, mean rotation error, and mean translation error.…”

Section: B Resultsmentioning

confidence: 99%

“…Some recent progress has been made to address the problem of category-level 6D pose estimation [11]- [16]. Besides the mentioned work on category-level pose estimation for single frames such as NOCS [11], CASS [12], FS-Net [13] and the category-level tracking work 6-PACK [14], recently, Weng et al [15] propose a unified framework that can handle 9DoF category-level pose tracking for rigid object instances. More recently, Wen et al propose BundleTrack [16] which uses deep neural networks to extract and match keypoints, then pose graph optimization is used for pose tracking.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

iCaps: Iterative Category-level Object Pose and Shape Estimation

Deng¹,

Geng²,

Fox³

2022

Preprint

View full text Add to dashboard Cite

This paper proposes a category-level 6D object pose and shape estimation approach iCaps 1 , which allows tracking 6D poses of unseen objects in a category and estimating their 3D shapes. We develop a category-level auto-encoder network using depth images as input, where feature embeddings from the auto-encoder encode poses of objects in a category. The auto-encoder can be used in a particle filter framework to estimate and track 6D poses of objects in a category. By exploiting an implicit shape representation based on signed distance functions, we build a LatentNet to estimate a latent representation of the 3D shape given the estimated pose of an object. Then the estimated pose and shape can be used to update each other in an iterative way. Our category-level 6D object pose and shape estimation pipeline only requires 2D detection and segmentation for initialization. We evaluate our approach on a publicly available dataset and demonstrate its effectiveness. In particular, our method achieves comparably high accuracy on shape estimation.

show abstract

Section: B Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

iCaps: Iterative Category-level Object Pose and Shape Estimation

Deng¹,

Geng²,

Fox³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Both directions require the robot to learn about each object from scratch, no matter how similar the object is to those it has previously experienced. To address this issue, recent works have proposed to predict canonicalized object coordinates [32] for category-level articulated object pose estimation [3], [10]. However, such representation is designed specifically for articulated pose estimation and can't perform other tasks such as shape reconstruction or view synthesis.…”

Section: B Articulated Object Pose Estimationmentioning

confidence: 99%

“…• We show that the proposed representation can perform category-level articulated pose estimation through analysis-by-synthesis with only RGB inputs. To the best of our knowledge, existing works for this task all require depth inputs [3], [10], [11], [12].…”

Section: Introductionmentioning

confidence: 99%

CLA-NeRF: Category-Level Articulated Neural Radiance Field

Tseng¹,

Liao²,

Lin³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose CLA-NeRF -a Category-Level Articulated Neural Radiance Field that can perform view synthesis, part segmentation, and articulated pose estimation. CLA-NeRF is trained at the object category level using no CAD models and no depth, but a set of RGB images with ground truth camera poses and part segments. During inference, it only takes a few RGB views (i.e., few-shot) of an unseen 3D object instance within the known category to infer the object part segmentation and the neural radiance field. Given an articulated pose as input, CLA-NeRF can perform articulation-aware volume rendering to generate the corresponding RGB image at any camera pose. Moreover, the articulated pose of an object can be estimated via inverse rendering. In our experiments, we evaluate the framework across five categories on both synthetic and real-world data. In all cases, our method shows realistic deformation results and accurate articulated pose estimation. We believe that both few-shot articulated object rendering and articulated pose estimation open doors for robots to perceive and interact with unseen articulated objects.

show abstract

“…Modeling articulated objects has wide applications in multiple fields including virtual and augmented reality, ob-ject functional understanding, and robotic manipulation. To understand articulated objects, recent works propose to train deep networks for estimating per-part poses and the joint angle parameters of an object instance in a known category [41,83]. However, if we want to interact with the articulated object (e.g., open a laptop), estimating its static state is not sufficient.…”

Section: Introductionmentioning

confidence: 99%

A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation

Kortylewski

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent work has made significant progress on using implicit functions, as a continuous representation for 3D rigid object shape reconstruction. However, much less effort has been devoted to modeling general articulated objects. Compared to rigid objects, articulated objects have higher degrees of freedom, which makes it hard to generalize to unseen shapes. To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation. We assume no prior knowledge on part geometry, articulation status, joint type, joint axis, and joint location. With this disentangled continuous representation, we demonstrate that we can control the articulation input and animate unseen instances with unseen joint angles. Furthermore, we propose a Test-Time Adaptation inference algorithm to adjust our model during inference. We demonstrate our model generalize well to out-of-distribution and unseen data, e.g., partial point clouds and real-world depth images. Project page: https://jitengmu.github.io/A-SDF/.

show abstract

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

Cited by 9 publications

References 31 publications

iCaps: Iterative Category-level Object Pose and Shape Estimation

iCaps: Iterative Category-level Object Pose and Shape Estimation

CLA-NeRF: Category-Level Articulated Neural Radiance Field

A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation

Contact Info

Product

Resources

About