Learning Multiscale Deep Features and SVM Regressors for Adaptive RGB-T Saliency Detection

Ma, Yunpeng; Sun, Dengdi; Meng, Qianqian; Ding, Zhuanlian; Li, Chenglong

doi:10.1109/iscid.2017.92

Cited by 20 publications

(19 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Li et al [27] proposed a robust multi-task manifold ranking based RGB-T salient object detection method with cross-modality consistency. Ma et al [28] presented an adaptive RGB-T saliency detection method by learning multiscale deep CNN features and SVM regressors. In [49], a novel collaborative graph learning algorithm was presented for RGB-T image saliency detection.…”

Section: Rgb-t Salient Object Detectionmentioning

confidence: 99%

“…Unlike RGB cameras, thermal infrared cameras are a kind of passive sensors that capture the thermal infrared radiation emitted by all objects with a temperature above absolute zeros, meaning that thermal infrared images are invariant to illumination conditions [26]- [28]. As a result, when applied to salient object detection, thermal images tend to provide additional saliency cues to boost the saliency detection performance.…”

Section: Introductionmentioning

confidence: 99%

“…Although the experimental results demonstrated its performance superiority over the traditional RGB-induced saliency detection methods, using low-level hand-crafted features might be a bottleneck for further performance improvement in [27]. In [28], a deep CNNs based RGB-T salient object detection method was developed, in which the saliency map of each modality, i.e., RGB and thermal infrared, was independently induced at the first stage. Afterwards, the saliency map for each RGB or thermal infrared image was first independently induced by using the deep CNNs.…”

Section: Introductionmentioning

confidence: 99%

“…However, such a fusion at the saliency map level does not seem to well explore the complementary information/features across RGB and thermal images. In addition, the deep CNNs in [28] were pre-trained for image classification on ImageNet dataset [31], rather than for salient object detection, meaning that the saliency cues might not be well explored. [16]; (d) Saliency maps induced from the thermal images by [16]; (e) Saliency maps induced from the RGB-T images by [27]; (f) Ground truth.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, rather than integrating RGB-T information only at the saliency map level [28], which might be already too late, we propose a novel end-to-end deep neural network for RGB-T salient object detection by fusing multi-modal information at various stages. The proposed method turns the challenging RGB-T salient object detection into a CNN feature fusion problem, which covers the following three subproblems: 1) How to effectively extract the single-modal features from the input RGB or thermal infrared images; 2) How to fuse the extracted multi-modal features in a comprehensive way; and 3) How to infer the final saliency map using the fused features.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

181

View full text Add to dashboard Cite

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multimodal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the crossmodal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

show abstract

Section: Rgb-t Salient Object Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%