A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation

Zhou, Guangliang; Yan, Yi; Wang, Deming; Chen, Qijun

doi:10.1109/tmm.2020.3001533

Cited by 24 publications

(3 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these methods may be time-consuming due to expensive pose-processing steps and may not fully exploit spatial geometric structure information. In contrast, approaches like [20], [44], [45] used two separate branch networks to extract appearance and geometric features, followed by fusion for pose computation. While effective, these methods are sensitive to modality data loss.…”

Section: B Rgb-d Fusionmentioning

confidence: 99%

“…In recent advancements aimed at enhancing feature extraction, methods have increasingly integrated depth data as a complementary source alongside RGB images [20], [21], [44], [45]. By leveraging this fusion of RGB and depth information, these approaches have achieved state-of-theart results, notably on standard datasets such as Occluded-LINEMOD [25] or YCB-Video [24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Object Pose Estimation Using Color Images and Predicted Depth Maps

Hoang,

Xuan Tan,

Nguyen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The task of object pose estimation in computer vision heavily relies on both color (RGB) and depth (D) images to provide crucial appearance and geometric information, assisting algorithms in understanding occlusions and object geometry, thereby enhancing accuracy. However, the dependency on specialized sensors capable of capturing depth poses challenges in terms of cost and availability. Consequently, researchers are exploring methods to estimate object poses solely from RGB images. Nevertheless, this approach encounters difficulties in handling occlusions, discerning object geometry, and resolving ambiguities arising from similar color or texture patterns. This paper introduces a novel geometry-aware method for object pose estimation utilizing RGB images as input to determine the poses of multiple object instances. Our approach leverages both depth and color images during training but only relies on color images during inference. Departing from traditional depth sensors, our method computes predicted point clouds directly from estimated depth images derived from RGB inputs. A key innovation lies in the formulation of a multi-scale fusion module adept at seamlessly integrating features extracted from RGB images with those inferred from the predicted point clouds. This fusion process significantly fortifies the pose estimation pipeline by harnessing the strengths of both modalities, resulting in notably improved object poses. Extensive experimentation demonstrates that our approach markedly outperforms state-of-the-art RGB-based methods on Occluded-LINEMOD and YCB-Video datasets. Moreover, our method achieves competitive results compared to RGB-D approaches that necessitate both RGB and depth data from physical sensors.

show abstract

Section: B Rgb-d Fusionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Object Pose Estimation Using Color Images and Predicted Depth Maps

Hoang,

Xuan Tan,

Nguyen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…But they neglect the help of geometry information for RGB representation learning, the BEV ignore the pitch and roll of object pose, and the regular 2D CNN is not good at contiguous geometry reasoning either. Instead, [70,49,66,74,65] extract features…”

Section: Pose Estimation With Rgb-d Datamentioning

confidence: 99%

FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation

He¹,

Huang²,

Fan³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work, we present FFB6D, a Full Flow Bidirectional fusion network designed for 6D pose estimation from a single RGBD image. Our key insight is that appearance information in the RGB image and geometry information from the depth image are two complementary data sources, and it still remains unknown how to fully leverage them. Towards this end, we propose FFB6D, which learns to combine appearance and geometry information for representation learning as well as output representation selection. Specifically, at the representation learning stage, we build bidirectional fusion modules in the full flow of the two networks, where fusion is applied to each encoding and decoding layer. In this way, the two networks can leverage local and global complementary information from the other one to obtain better representations. Moreover, at the output representation stage, we designed a simple but effective 3D keypoints selection algorithm considering the texture and geometry information of objects, which simplifies keypoint localization for precise pose estimation. Experimental results show that our method outperforms the state-of-the-art by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/FFB6D.git.

show abstract