T6D-Direct: Transformers for Multi-object 6D Pose Direct Regression

Amini, Arash; Periyasamy, Arul Selvam; Behnke, Sven

doi:10.1007/978-3-030-92659-5_34

Cited by 18 publications

(7 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, few attempts to use Transformers have been made in 6D object pose estimation. Recently, some studies showed that Transformers have a competitive performance in 6D object pose estimation as well [28][29][30][31]51,52]. PoET [28] is a Transformerbased framework that takes a single RGB image as input and estimates the 6D poses for all objects present in the image without object 3D CAD models.…”

Section: Transformer-based Methodsmentioning

confidence: 99%

FusionNet: An End-to-End Hybrid Model for 6D Object Pose Estimation

Ye,

Park

2023

Electronics

View full text Add to dashboard Cite

In this study, we propose a hybrid model for Perspective-n-Point (PnP)-based 6D object pose estimation called FusionNet that takes advantage of convolutional neural networks (CNN) and Transformers. CNN is an effective and potential tool for feature extraction, which is considered the most popular architecture. However, CNN has difficulty in capturing long-range dependencies between features, and most CNN-based models for 6D object pose estimation are bulky and heavy. To address these problems, we propose a lighter-weight CNN building block with attention, design a Transformer-based global dependency encoder, and integrate them into a single model. Our model is able to extract dense 2D–3D point correspondences more accurately while significantly reducing the number of model parameters. Followed with a PnP header that replaces the PnP algorithm for general end-to-end pose estimation, our model showed better or highly competitive performance in pose estimation compared with other state-of-the-art models in experiments on the LINEMOD dataset.

show abstract

Section: Transformer-based Methodsmentioning

confidence: 99%

FusionNet: An End-to-End Hybrid Model for 6D Object Pose Estimation

Ye,

Park

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Pose estimation problem can be formulated as direct classification [14], regression [40], 2D-3D correspondences [41], or 3D-3D correspondences [42]. The following sections discuss various methods, along with some of the common loss functions associated with them.…”

Section: Problem Formulationmentioning

confidence: 99%

“…These classification methods usually outperform these methods in pose estimation. Amini et al [40] formulate pose estimation as a regression. In this paper the loss is formulated as a distance of the projected points in 2D as follow:…”

Section: Regressionmentioning

confidence: 99%

“…The recently proposed language model, Transformer [64], attracts lots of attention in all areas. T6D-Direct [40], inspiring from Transformer-based object detection model DETR [65], proposed an approach which formulate 6D object pose direct regression as a set prediction problem. It is a single-stage direct method that directly estimates multi-object poses and bounding boxes.…”

Section: Keypoint-basedmentioning

confidence: 99%

See 1 more Smart Citation

Review on 6D Object Pose Estimation With the Focus on Indoor Scene Understanding

Nejatishahidin¹,

Pooya²

2022

AAIML

View full text Add to dashboard Cite

6D object pose estimation problem has been extensively studied in the field of Computer Vision and Robotics. It has wide range of applications such as robot manipulation, augmented reality, and 3D scene understanding. With the advent of Deep Learning, many breakthroughs have been made; however, approaches continue to struggle when they encounter unseen instances, new categories, or real-world challenges such as cluttered backgrounds and occlusions. In this study, we will explore the available methods based on input modality, problem formulation, and whether it is a category-level or instance-level approach. As a part of our discussion, we will focus on how 6D object pose estimation can be used for understanding 3D scenes.

show abstract

“…Benefiting from the powerful long-range modeling capability of multi-head self-attention (MSA) module, vision transformer (ViT) (Dosovitskiy et al 2020) and its variants (Touvron et al 2021;Liu et al 2021a) have achieved promising performance against the convolution neural networks in a variety of computer vision tasks (Amini, Periyasamy, and Behnke 2021;He et al 2021). Although ViTs provide an architecture with better feature representation, the high computational cost and massive parameters restrict their application in resource-limited devices (Chen et al 2021;Chuanyang et al 2022).…”

Section: Introductionmentioning

confidence: 99%

BVT-IMA: Binary Vision Transformer with Information-Modified Attention

Wang,

Luo,

Xie

et al. 2024

AAAI

View full text Add to dashboard Cite

As a compression method that can significantly reduce the cost of calculations and memories, model binarization has been extensively studied in convolutional neural networks. However, the recently popular vision transformer models pose new challenges to such a technique, in which the binarized models suffer from serious performance drops. In this paper, an attention shifting is observed in the binary multi-head self-attention module, which can influence the information fusion between tokens and thus hurts the model performance. From the perspective of information theory, we find a correlation between attention scores and the information quantity, further indicating that a reason for such a phenomenon may be the loss of the information quantity induced by constant moduli of binarized tokens. Finally, we reveal the information quantity hidden in the attention maps of binary vision transformers and propose a simple approach to modify the attention values with look-up information tables so that improve the model performance. Extensive experiments on CIFAR-100/TinyImageNet/ImageNet-1k demonstrate the effectiveness of the proposed information-modified attention on binary vision transformers.

show abstract

T6D-Direct: Transformers for Multi-object 6D Pose Direct Regression

Cited by 18 publications

References 31 publications

FusionNet: An End-to-End Hybrid Model for 6D Object Pose Estimation

FusionNet: An End-to-End Hybrid Model for 6D Object Pose Estimation

Review on 6D Object Pose Estimation With the Focus on Indoor Scene Understanding

BVT-IMA: Binary Vision Transformer with Information-Modified Attention

Contact Info

Product

Resources

About