RGB-T Image Saliency Detection via Collaborative Graph Learning

Tu, Zhengzheng; Xia, Tian; Li, Chenglong; Wang, Xiaoxiao; Ma, Yan; Tang, Jin

doi:10.1109/tmm.2019.2924578

Cited by 157 publications

(91 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare the proposed multi-modality saliency detection model with 11 state-of-the-art methods on two datasets, including 5 deep learning based RGB saliency detection methods (BMPM [10], DSS [11], Amulet [12], UCF [61], and CPD [62]), 4 RGB-T saliency detection approaches (MRCMC [56], MFSR [35], CGL [57], and FMCF 1 [36] ) and 2 latest RGB-D salient object detection models (PDNet [31] and TSAA [32]). To better verify the superiority of the proposed model, as shown in Table I, these state-of-the-art methods are compared under different settings of the input modality, i.e., only taking RGB images, only taking thermal images and taking RGB-T images as inputs, respectively.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…When compared with multi-level features learned from CNN, these hand-crafted features are less discriminative. Then Tu et al [57] posed saliency detection to a graph learning problem and utilized hierarchical deep features to jointly learn saliency. Ma et al [35] adaptively incorporated RGB and thermal saliency maps inferred from deep convolutional neural networks.…”

Section: B Multi-modality Salient Object Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Feature Fusion for RGB-T Salient Object Detection

Zhang

Xiao

Huang

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

101

View full text Add to dashboard Cite

While many RGB-based saliency detection algorithms have recently shown the capability of segmenting salient objects from an image, they still suffer from unsatisfactory performance when dealing with complex scenarios, insufficient illumination or occluded appearances. To overcome this problem, this paper studies RGB-T saliency detection, where we take advantage of thermal modality's robustness against illumination and occlusion. To achieve this goal, we revisit feature fusion for mining intrinsic RGB-T saliency patterns and propose a novel deep feature fusion network, which consists of the multiscale, multi-modality, and multi-level feature fusion modules. Specifically, the multi-scale feature fusion module captures rich contexture features from each modality feature, while the multimodality and multi-level feature fusion modules integrate complementary features from different modality features and different level of features, respectively. To demonstrate the effectiveness of the proposed approach, we conduct comprehensive experiments on the RGB-T saliency detection benchmark. The experimental results demonstrate that our approach outperforms other stateof-the-art methods and the conventional feature fusion modules by a large margin.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: B Multi-modality Salient Object Detectionmentioning

confidence: 99%

Revisiting Feature Fusion for RGB-T Salient Object Detection

Zhang

Xiao

Huang

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

101

View full text Add to dashboard Cite

show abstract

“…To validate the proposed RGB-T salient detection model, we compare our model with 10 SOTA methods, which are further divided into three types, i.e., (1)RGB salient object detection methods: PoolNet [39], R3Net [40], and CPDNet [41]; (2) RGB-D salient object detection methods: AFNet [45], TSAA [46], PDNet [47], and SSRC [48]; and (3) RGB-T salient object detection methods: MFSR [28], GCL [49], and MRCM [27]. For fair comparisons, we modify these RGB and RGB-D salient object detection methods for RGB-T saliency detection.…”

Section: Comparison With the State-of-the-art Methodsmentioning

confidence: 99%

“…Ma et al [28] presented an adaptive RGB-T saliency detection method by learning multiscale deep CNN features and SVM regressors. In [49], a novel collaborative graph learning algorithm was presented for RGB-T image saliency detection. Specifically, superpixels were taken as graph nodes, and hierarchical deep features were collaboratively used to jointly learn the graph affinity and node saliency in a unified optimization framework.…”

Section: Rgb-t Salient Object Detectionmentioning

confidence: 99%

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

180

View full text Add to dashboard Cite

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multimodal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the crossmodal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

show abstract

“…Therefore, thermal images can provide supplementary information to improve salient object detection when images of salient objects suffer from varying light, glare, or shadows. Some RGB-T models [197][198][199][200][201][202][203][204][205] and datasets (VT821 [199], VT1000 [203], and VT5000 [205]) have already been proposed over the past few years. Like for RGB-D salient object detection, the key aim of RGB-T salient object detection is to fuse RGB and thermal infrared images and exploit the correlations between the two modalities.…”

Section: Extension To Rgb-tmentioning

confidence: 99%

RGB-D salient object detection: A survey

et al. 2021

View full text Add to dashboard Cite

Salient object detection, which simulates human visual perception in locating the most significant object(s) in a scene, has been widely applied to various computer vision tasks. Now, the advent of depth sensors means that depth maps can easily be captured; this additional spatial information can boost the performance of salient object detection. Although various RGB-D based salient object detection models with promising performance have been proposed over the past several years, an in-depth understanding of these models and the challenges in this field remains lacking. In this paper, we provide a comprehensive survey of RGB-D based salient object detection models from various perspectives, and review related benchmark datasets in detail. Further, as light fields can also provide depth maps, we review salient object detection models and popular benchmark datasets from this domain too. Moreover, to investigate the ability of existing models to detect salient objects, we have carried out a comprehensive attribute-based evaluation of several representative RGB-D based salient object detection models. Finally, we discuss several challenges and open directions of RGB-D based salient object detection for future research. All collected models, benchmark datasets, datasets constructed for attribute-based evaluation, and related code are publicly available at https://github.com/taozh2017/RGBD-SODsurvey.

show abstract

RGB-T Image Saliency Detection via Collaborative Graph Learning

Cited by 157 publications

References 36 publications

Revisiting Feature Fusion for RGB-T Salient Object Detection

Revisiting Feature Fusion for RGB-T Salient Object Detection

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

RGB-D salient object detection: A survey

Contact Info

Product

Resources

About