Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction

Zhou, Wujie; Lv, Yiliang; Lei, Jingsheng; Yu, Lu

doi:10.1109/tsmc.2019.2957386

Cited by 118 publications

(32 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to [26] and [62], SOS [28], and fixation prediction [63], [64] have strong connections with SOD. Besides, salient edge detection is also highly relevant to salient object information.…”

Section: ) Auxiliary Networkmentioning

confidence: 97%

DANet: Dynamic Salient Object Detection Networks Leveraging Auxiliary Information

et al. 2021

View full text Add to dashboard Cite

Based on Fully Convolutional Networks, recent salient object detection (SOD) methods achieve impressive results. Some studies improve the classical SOD frameworks by utilizing auxiliary information like fixation points, salient numbers and salient edges. These works adopt auxiliary information by embedding sub-networks into the main network. However, how to incorporate auxiliary information regardless of the specific structure with less coupling is unexplored in SOD. In this paper, we present DANet, a new Dynamic network to leverage arbitrary Auxiliary information for SOD. The proposed framework consists of 1) a Dynamic Weight Generator (DWG) which converts arbitrary auxiliary features into dynamic weights, 2) a Dynamic Bridge Block (DBB) which uses dynamic weight convolution to incorporate auxiliary information from DWG and then refines the fused features, and 3) a two-step training strategy to alleviate the side effect caused by drastic changes among different input images. Extensive experiments demonstrate the effect of different auxiliary information and the proposed framework is a universal method to improve SOD with auxiliary information. Comparison experiments show that DANet achieves state-of-the-art performance without any pre-processing and post-processing.

show abstract

Section: ) Auxiliary Networkmentioning

confidence: 97%

DANet: Dynamic Salient Object Detection Networks Leveraging Auxiliary Information

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Meanwhile, the research on RGBD saliency detection [43,44] has also been pushed forward significantly in recent decades. Many RGBD saliency models exist, including heuristic models [5,[13][14][15][16][17][18][19][20][21][22][23] and deep learning-based models [24][25][26][27][28][29][30][31][32][33][34][35]45], which have achieved encouraging performance. Following, we introduce some of the existing RGBD saliency models.…”

Section: Related Workmentioning

confidence: 99%

AFI‐Net: Attention‐Guided Feature Integration Network for RGBD Saliency Detection

Zhao

Sun

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

This article proposes an innovative RGBD saliency model, that is, attention-guided feature integration network, which can extract and fuse features and perform saliency inference. Specifically, the model first extracts multimodal and level deep features. Then, a series of attention modules are deployed to the multilevel RGB and depth features, yielding enhanced deep features. Next, the enhanced multimodal deep features are hierarchically fused. Lastly, the RGB and depth boundary features, that is, low-level spatial details, are added to the integrated feature to perform saliency inference. The key points of the AFI-Net are the attention-guided feature enhancement and the boundary-aware saliency inference, where the attention module indicates salient objects coarsely, and the boundary information is used to equip the deep feature with more spatial details. Therefore, salient objects are well characterized, that is, well highlighted. The comprehensive experiments on five challenging public RGBD datasets clearly exhibit the superiority and effectiveness of the proposed AFI-Net.

show abstract

“…With the rapid development in the consumer electronic industry, various RGB-D applications and services have become increasingly popular for enhanced user experience [1][2][3][4][5][6]. e RGB-D image processing technologies for RGB-D applications and services can be further improved by developing better models of RGB-D perception [7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Deep Multimodal Fusion Autoencoder for Saliency Prediction of RGB‐D Images

Huang

Zhou

Fang

2021

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

In recent years, the prediction of salient regions in RGB-D images has become a focus of research. Compared to its RGB counterpart, the saliency prediction of RGB-D images is more challenging. In this study, we propose a novel deep multimodal fusion autoencoder for the saliency prediction of RGB-D images. The core trainable autoencoder of the RGB-D saliency prediction model employs two raw modalities (RGB and depth/disparity information) as inputs and their corresponding eye-fixation attributes as labels. The autoencoder comprises four main networks: color channel network, disparity channel network, feature concatenated network, and feature learning network. The autoencoder can mine the complex relationship and make the utmost of the complementary characteristics between both color and disparity cues. Finally, the saliency map is predicted via a feature combination subnetwork, which combines the deep features extracted from a prior learning and convolutional feature learning subnetworks. We compare the proposed autoencoder with other saliency prediction models on two publicly available benchmark datasets. The results demonstrate that the proposed autoencoder outperforms these models by a significant margin.

show abstract

Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction

Cited by 118 publications

References 42 publications

DANet: Dynamic Salient Object Detection Networks Leveraging Auxiliary Information

DANet: Dynamic Salient Object Detection Networks Leveraging Auxiliary Information

AFI‐Net: Attention‐Guided Feature Integration Network for RGBD Saliency Detection

Deep Multimodal Fusion Autoencoder for Saliency Prediction of RGB‐D Images

Contact Info

Product

Resources

About