Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

Gao, Wei; Liao, Guibiao; Ma, Siwei; Li, Ge; Liang, Yongsheng; Wang, Lin

doi:10.1109/tcsvt.2021.3082939

Cited by 131 publications

(31 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MIDD [62] proposes multi-interactive dual-decoder to integrate the multi-level interactions of dual modalities and global contexts. MMNet [63] simulates visual color stage doctrine to fuse cross-modal features in stages, and designs bi-directional multi-scale decoder to capture both local and global information. CGFNet [64] adopts the guidance manner of one modality on the other modality to fuse two modalities.…”

Section: B Rgb-t Salient Object Detectionmentioning

confidence: 99%

“…For RGB-D SOD, our model is compared with several SOTA RGB-D SOD algorithms, including D3Net [78], ASIF-Net [36], ICNet [89], DCMF [52], DRLF [90], SSF [43], SSMA [38], A2dele [46], UC-Net [91], JL-DCF [92], CoNet [44], DANet [81], EBFSP [93],CDNet [94], HAINet [95], RD3D [49], DSA2F [48], MMNet [63] and VST [6]. To ensure the fairness of the comparison results, the saliency maps of the evaluation are provided by the authors or generated by running source codes.…”

Section: Comparisons With Sotas 1) Rgb-d Sodmentioning

confidence: 99%

“…For RGB-T SOD, our model is compared with some SOTA RGB-T SOD algorithms, including MTMR [56], M3S-NIR [55], SGDL [58], ADF [59], ECFFNet [11], MIDD [62], MMNet [63], CSRNet [65], CGFNet [64]. To ensure the fairness of the comparison results, the saliency maps of the evaluation are provided by the authors or generated by running source codes.…”

Section: ) Rgb-t Sodmentioning

confidence: 99%

See 2 more Smart Citations

SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection

Liu

Tan

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

203

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) are good at extracting contexture features within certain receptive fields, while transformers can model the global long-range dependency features. By absorbing the advantage of transformer and the merit of CNN, Swin Transformer shows strong feature representation ability. Based on it, we propose a cross-modality fusion model, SwinNet, for RGB-D and RGB-T salient object detection.It is driven by Swin Transformer to extract the hierarchical features, boosted by attention mechanism to bridge the gap between two modalities, and guided by edge information to sharp the contour of salient object. To be specific, two-stream Swin Transformer encoder first extracts multi-modality features, and then spatial alignment and channel re-calibration module is presented to optimize intra-level cross-modality features. To clarify the fuzzy boundary, edge-guided decoder achieves inter-level cross-modality fusion under the guidance of edge features. The proposed model outperforms the state-of-theart models on RGB-D and RGB-T datasets, showing that it provides more insight into the cross-modality complementarity task.https://github.com/liuzywen/SwinNet

show abstract

Section: B Rgb-t Salient Object Detectionmentioning

confidence: 99%

Section: Comparisons With Sotas 1) Rgb-d Sodmentioning

confidence: 99%

Section: ) Rgb-t Sodmentioning

confidence: 99%

See 1 more Smart Citation

SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection

Liu

Tan

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

203

View full text Add to dashboard Cite

show abstract

“…Despite some improvements, most feature fusion based RGB-D SOD models mentioned above mainly focus on capturing the complementary information within the multi-modality input images, while ignoring the impacts of image qualities on the representation ability of fused features, thus degrading the subsequent saliency detection performance. Recently, some studies have been carried out on the disturbing problem caused by the low-quality images [13], [19]- [21], [54]- [57]. For example, Zhao et al [19] designed a contrast enhancement module with contrast prior information to enhance the quality of depth images, thus boosting the saliency detection performance.…”

Section: B Rgb-d Salient Object Detectionmentioning

confidence: 99%

“…[56] modeled a task-orientated depth potentiality perception module to weaken the contamination from unreliable depth information. Gao et al [57] used the content-based spatial attention to select the important response of intra-modal information.…”

Section: B Rgb-d Salient Object Detectionmentioning

confidence: 99%