CSA6D: Channel-Spatial Attention Networks for 6D Object Pose Estimation

Abstract

Abstract6D object pose estimation plays a crucial role in robotic manipulation and grasping tasks. The aim to estimate the 6D object pose from RGB or RGB-D images is to detect objects and estimate their orientations and translations relative to the given canonical models. RGB-D cameras provide two sensory modalities: RGB and depth images, which could benefit the estimation accuracy. But the exploitation of two different modality sources remains a challenging issue. In this paper, inspired by recent works on attention networks that could focus on important regions and ignore unnecessary information, we propose a novel network: Channel-Spatial Attention Network (CSA6D) to estimate the 6D object pose from RGB-D camera. The proposed CSA6D includes a pre-trained 2D network to segment the interested objects from RGB image. Then it uses two separate networks to extract appearance and geometrical features from RGB and depth images for each segmented object. Two feature vectors for each pixel are stacked together as a fusion vector which is refined by an attention module to generate a aggregated feature vector. The attention module includes a channel attention block and a spatial attention block which can effectively leverage the concatenated embeddings into accurate 6D pose prediction on known objects. We evaluate proposed network on two benchmark datasets YCB-Video dataset and LineMod dataset and the results show it can outperform previous state-of-the-art methods under ADD and ADD-S metrics. Also, the attention map demonstrates our proposed network searches for the unique geometry information as the most likely features for pose estimation. From experiments, we conclude that the proposed network can accurately estimate the object pose by effectively leveraging multi-modality features.

A Robust Convolutional Neural Network for 6D Object Pose Estimation from RGB Image with Distance Regularization Voting Loss

“…He et al [12] incorporate a channel-level attention module for the adaptive feature fusion into U-Net and calculate distances between pixels and keypoints using prior distance augmented loss. Another related architecture based on the channel spatial attention network (CSA6D) is proposed by Chen and Gu [40] to estimate the 6D object pose from RGB-D images.…”

“…Here, we compare our results with 6D pose estimation approaches using a single RGB image, which are state of the art in this research area. e comparisons have been carried out against PVNet [8], DPVL [11], ASPP-DF-PVNet with L+ loss [7], and PDAL-AFAM approach of He et al (2021) [12] and some previous approaches such as PoseCNN [5], SSD6D [1], YOLO6D [3], BB8 [29], CDPN [32], DPOD [31], Pix2Pose [33], and CSA6D [40]. e results are evaluated using ADD (-S) and 2D-Projection metrics on LINEMOD and occlusion LINEMOD datasets.…”

“…We include only those results for comparisons that are provided by other methods as 2D projection-based results are not reported by some methods, so we do not include those in Tables 3 and 4. CSA6D [40] reported only 2D projection-based results on the LINEMOD dataset, so we only include those. Table 3 shows the comparison of our method with a number of other methods for pose estimation on the LINEMOD dataset concerning the 2D projection metric.…”

Joint Hand and Object Pose Estimation from a Single RGB Image using High‐level 2D Constraints

“…There has been great progress in reconstructing or estimating the pose of a single hand [KS12,GRL*19,IMB*18,CCY*21,ZLM*19] or objects [HHFS19, KMT*17, PLH*19, ZSI19, LF20, ZHMW22, LZXQ21, YJLF22, CG22, ZBB21, SHCM21] alone over recent decades. Lacking good datasets labeling hands and objects together, early work on hand‐object interaction focused on recovering either the hand [RKK09, RKI*14] or object [TG15] pose in a interaction.…”

PointGS: Bridging and fusing geometric and semantic space for 3D point cloud analysis

“…With recent advances in 3D scanning technologies, it becomes convenient to obtain 3D raw data. As the fundamental 3D representation, point cloud has attracted extensive attention for various 3D applications [1,2]. Recently, researchers focus on exploiting Convolution Neural Networks (CNNs) to process 3D point cloud, which can be generally categorized into three types: projectionbased methods [3,4,5], voxelization-based methods [6,7], and point-based methods [8,9,10,11].…”