PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

Nguyen, Van; Du, Yuming; Xiao, Yang; Ramamonjisoa, Michaël; Lepetit, Vincent

doi:10.1109/3dv57658.2022.00063

Cited by 11 publications

(8 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we first describe our experimental setup in Section 4.1, and then compare our method to others ( [21,22,27,52] on both seen and unseen object categories in Section 4.2. Section 4.3 reports an evaluation of the robustness to partial occlusions.…”

Section: Methodsmentioning

confidence: 99%

“…Dataset. To the best of our knowledge, we are the first method addressing the problem of object pose estimation from a single image when the object belongs to a category not seen during training: PIZZA [27] evaluated on the DeepIM refinement benchmark, which is made of pairs of images with a small relative pose; SSVE [22] and ViewNet [21] evaluated only on objects from categories seen during training.…”

Section: Methodsmentioning

confidence: 99%

“…Baselines. We compare our work with all previous methods (to the best of our knowledge) that aim to predict a pose from a single view: PIZZA [27], a regressionbased approach that directly predicts the relative pose, and SSVE [22] and ViewNet [21], which employ semisupervised and self-supervised techniques to treat viewpoint estimation as an image reconstruction problem using conditional generation.…”

Section: Methodsmentioning

confidence: 99%

“…Yet, it yields accurate pose estimates with more than 65% accuracy on average on unseen instances or categories. Our work is also related to motion prediction between 2 images, such as [27,49], which predict the camera displacement between two images. Our comparisons to the closest work to ours [27] show that we significantly outperform this approach.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Nguyen

Hu²,

Xiao

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

ReferenceQuery Predicted pose Pose distribution Reference Query Predicted pose Pose distribution Figure 1. Given as input a single reference view of a novel object, our method predicts the relative 3D pose of a query view and its ambiguities. We visualize the predicted pose by rendering the object from this pose, but the 3D model is only used for visualization purposes, not as input to our method. Our method works by estimating a probability distribution over the space of 3D poses, visualized here on a sphere centered on the object. We use the canonical pose of the 3D model to visualize this distribution, but not as input to our method. From this distribution, we can also identify the pose ambiguities: For example, in the case of the bottle, any pose with the same pitch and roll is possible; in the case of the mug, a range of poses are possible as the handle is not visible in the query image. Our method is also robust to partial occlusions, as shown on the clock hidden in part by a rectangle in the query image.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Nguyen

Hu²,

Xiao

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…These templates can either be generated synthetically as in [14] or with a few real images as in FS6D [8]. The same approach have also been used for tracking of unknown objects [13]. MegaPose6D [12] is a notable example where the network is trained on a huge dataset with 2 million images.…”

Section: Related Workmentioning

confidence: 99%

GP3D: Generalized Pose Estimation in 3D Point Clouds: A case study on bin picking

Hagelskjær¹

2023

Preprint

View full text Add to dashboard Cite

In this paper, we present GP3D, a novel network for generalized pose estimation in 3D point clouds. The method generalizes to new objects by using both the scene point cloud and the object point cloud with keypoint indexes as input. The network is trained to match the object keypoints to scene points.To address the pose estimation of novel objects we also present a new approach for training pose estimation. The typical solution is a single model trained for pose estimation of a specific object in any scenario. This has several drawbacks: training a model for each object is time-consuming, energy consuming, and by excluding the scenario information the task becomes more difficult.In this paper, we present the opposite solution; a scenariospecific pose estimation method for novel objects that do not require retraining. The network is trained on 1500 objects and is able to learn a generalized solution. We demonstrate that the network is able to correctly predict novel objects, and demonstrate the ability of the network to perform outside of the trained class. We believe that the demonstrated method is a valuable solution for many real-world scenarios. Code and trained network will be made available after publication.

show abstract

Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality

Segas

Mick

Leconte

et al. 2022

Preprint

View full text Add to dashboard Cite

Impressive progresses are being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in arm amputees based on proximal stump motion and knowledge of the movement goal. This control was validated on 29 participants, including 7 above-elbow amputees, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.

show abstract

PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

Cited by 11 publications

References 51 publications

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

GP3D: Generalized Pose Estimation in 3D Point Clouds: A case study on bin picking

Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality

Contact Info

Product

Resources

About