Bi-Directional Progressive Guidance Network for RGB-D Salient Object Detection

Yang, Yang; Qin, Qi; Luo, Yongjiang; Liu, Yi; Zhang, Qiang; Han, Jungong

doi:10.1109/tcsvt.2022.3144852

Cited by 50 publications

(11 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The potential of the self-attention mechanism in SOD was first recognized by Liu et al [9] and Zhang et al [10]. Subsequent studies [32]- [34] have further expanded the application of self-attention in SOD tasks. In our work, we introduce a novel attention mechanism called Mixed-frequency Attention, which employs one attention head to concentrate solely on saliency information while another focuses on the interaction between edge and saliency information.…”

Section: Attention Mechanism In Salient Object Detectionmentioning

confidence: 98%

Spectrum-Driven Mixed-Frequency Network for Hyperspectral Salient Object Detection

Liu,

Xu,

Chen

et al. 2024

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Hyperspectral salient object detection (HSOD) aims to detect spectrally salient objects in hyperspectral images (HSIs). However, existing methods inadequately utilize spectral information by either converting HSIs into false-color images or converging neural networks with clustering. We propose a novel approach that fully leverages the spectral characteristics by extracting two distinct frequency components from the spectrum: low-frequency Spectral Saliency and high-frequency Spectral Edge. The Spectral Saliency approximates the region of salient objects, while the Spectral Edge captures edge information of salient objects. These two complementary components, crucial for HSOD, are derived by computing from the inter-layer spectral angular distance of the Gaussian pyramid and the intra-neighborhood spectral angular gradients, respectively. To effectively utilize this dual-frequency information, we introduce a novel lightweight Spectrum-driven Mixed-frequency Network (SMN). SMN incorporates two parameter-free plug-and-play operators, namely Spectral Saliency Generator and Spectral Edge Operator, to extract the Spectral Saliency and Spectral Edge components from the input HSI independently. Subsequently, the Mixed-frequency Attention module, comprised of two frequencydependent heads, intelligently combines the embedded features of edge and saliency information, resulting in a mixed-frequency feature representation. Furthermore, a saliency-edge-aware decoder progressively scales up the mixed-frequency feature while preserving rich detail and saliency information for accurate salient object prediction. Extensive experiments conducted on the HS-SOD benchmark and our custom dataset HSOD-BIT demonstrate that our SMN outperforms state-of-the-art methods regarding HSOD performance. Code and dataset will be available at https://github.com/laprf/SMN.

show abstract

Section: Attention Mechanism In Salient Object Detectionmentioning

confidence: 98%

Spectrum-Driven Mixed-Frequency Network for Hyperspectral Salient Object Detection

Liu,

Xu,

Chen

et al. 2024

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…With the emergence of RGB-D sensors, researchers turn to investigating how to assist image processing with 3D information [18]- [21], including depth images and point cloud. Wang et al [22] introduce D-CNN and average pooling to improve the capability of handling geometric information from depth images.…”

Section: A Semantic Segmentationmentioning

confidence: 99%

TROSD: A New RGB-D Dataset for Transparent and Reflective Object Segmentation in Practice

Sun

Zhang

Yang

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Transparent and reflective objects are omnipresent in our daily life, but their unique visual and optical characteristics are notoriously challenging even for state-of-the-art deep networks of semantic segmentation. To alleviate this challenge, we construct a new large-scale real-world RGB-D dataset called TROSD, which is more comprehensive than existing datasets for transparent and reflective object segmentation. Our TROSD dataset contains 11,060 RGB-D images with three semantic classes in terms of transparent objects, reflective objects, and others, covering a variety of daily scenes. Together with the dataset, we also introduce a novel network (TROSNet) as a high-standard baseline to assist other researchers to develop and benchmark their algorithms of transparent and reflective object segmentation. Moreover, extensive experiments also clearly show that the proposed TROSD dataset has an excellent capacity to facilitate the development of semantic segmentation algorithms with strong generalizability.

show abstract

“…Early fusion [45], middle fusion [46]- [49] and late fusion [50]- [52] are three classic frameworks. The performance of RGB-D SOD can be improved using an attention mechanism [53], edge guidance [54], [55], depth calibration [56], depth estimation [57], [58], depth quality assessments [59], [60], 3D convolution [61], deformable convolution [62], automatic architecture search [63], uncertainty distribution [64], and bi-directional guidance [65], [66]. To reduce the influence of poor depth images, Cong et al [67] use salient seed diffusion with a depth constraint and introduce a depth confidence weight [68].…”

Section: B Two-modality Salient Object Detectionmentioning

confidence: 99%

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Tang

Liu

Tan

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intrafeature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet

show abstract

Bi-Directional Progressive Guidance Network for RGB-D Salient Object Detection

Cited by 50 publications

References 80 publications

Spectrum-Driven Mixed-Frequency Network for Hyperspectral Salient Object Detection

Spectrum-Driven Mixed-Frequency Network for Hyperspectral Salient Object Detection

TROSD: A New RGB-D Dataset for Transparent and Reflective Object Segmentation in Practice

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Contact Info

Product

Resources

About