Robust Deep Co-Saliency Detection With Group Semantic and Pyramid Attention

Zha, Zheng-Jun; Wang, Chong; Liu, Dong; Xie, Hongtao; Zhang, Yongdong

doi:10.1109/tnnls.2020.2967471

Cited by 34 publications

(14 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PSPNet [25] utilizes the average pooling operation to exploit global context information over four different pyramid scales. Moreover, attention mechanisms have been employed in many works [26]- [28] to capture multi-scale interactions. For example, SENet [29] applies channel-based attention to modulate the weight of CNN channels selectively.…”

Section: B Local and Global Contextmentioning

confidence: 99%

Denoised Non-Local Neural Network for Semantic Segmentation

Song¹,

Li²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

The non-local network has become a widely used technique for semantic segmentation, which computes an attention map to measure the relationships of each pixel pair. However, most of the current popular non-local models tend to ignore the phenomenon that the calculated attention map appears to be very noisy, containing inter-class and intra-class inconsistencies, which lowers the accuracy and reliability of the non-local methods. In this paper, we figuratively denote these inconsistencies as attention noises and explore the solutions to denoise them. Specifically, we inventively propose a Denoised Non-Local Network (Denoised NL), which consists of two primary modules, i.e., the Global Rectifying (GR) block and the Local Retention (LR) block, to eliminate the inter-class and intra-class noises respectively. First, GR adopts the class-level predictions to capture a binary map to distinguish whether the selected two pixels belong to the same category. Second, LR captures the ignored local dependencies and further uses them to rectify the unwanted hollows in the attention map. The experimental results on two challenging semantic segmentation datasets demonstrate the superior performance of our model. Without any external training data, our proposed Denoised NL can achieve the stateof-the-art performance of 83.5% and 46.69% mIoU on Cityscapes and ADE20K, respectively.

show abstract

Section: B Local and Global Contextmentioning

confidence: 99%

Denoised Non-Local Neural Network for Semantic Segmentation

Song¹,

Li²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…where U ∈ R N×m is a subspace matrix whose columns are orthonormal, m is the number of columns of U, and U i stands for the ith row of U. The coefficient vector v ∈ R m×1 is the low-dimensional representation of frame c in the subspace spanned by the rows of U. s ∈ R N×1 is the saliency map obtained by some salient object detection algorithms, such as those in [43][44][45], and s i is the ith element of s. D = [D h , D v ] T is a difference matrix, and D h and D v are forward finite-difference operators in the horizontal and vertical directions, respectively. α, β and λ are the balancing parameters.…”

Section: Moving Object Modelingmentioning

confidence: 99%

Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering

Jia

et al. 2021

Sensors

View full text Add to dashboard Cite

Outdoor vision sensing systems often struggle with poor weather conditions, such as snow and rain, which poses a great challenge to existing video desnowing and deraining methods. In this paper, we propose a novel video desnowing and deraining model that utilizes the salience information of moving objects to address this problem. First, we remove the snow and rain from the video by low-rank tensor decomposition, which makes full use of the spatial location information and the correlation between the three channels of the color video. Second, because existing algorithms often regard sparse snowflakes and rain streaks as moving objects, this paper injects salience information into moving object detection, which reduces the false alarms and missed alarms of moving objects. At the same time, feature point matching is used to mine the redundant information of moving objects in continuous frames, and a dual adaptive minimum filtering algorithm in the spatiotemporal domain is proposed by us to remove snow and rain in front of moving objects. Both qualitative and quantitative experimental results show that the proposed algorithm is more competitive than other state-of-the-art snow and rain removal methods.

show abstract

“…In recent years, some researchers have extended deep learning techniques to co‐saliency detection algorithms and obtained good performance by co‐training common saliency objects [15–21]. Zhang et al.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, some researchers have extended deep learning techniques to co-saliency detection algorithms and obtained good performance by co-training common saliency objects [15][16][17][18][19][20][21]. Zhang et al [15][16][17] proposed a self-paced multiple-instance learning framework for co-saliency detection [17].…”

Section: Co-saliency Detection Algorithmmentioning

confidence: 99%

Consistent image processing based on co‐saliency

Ren

Hua

et al. 2021

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

In a group of images, the recurrent foreground objects are considered as the key objects in the group of images. In co-saliency detection, these are described as common saliency objects. The aim is to be able to naturally guide the user's gaze to these common salient objects. By guiding the user's gaze, users can easily find these common saliency objects without interference from other information. Therefore, a method is proposed for reducing user visual attention based on co-saliency detection. Through the co-saliency detection algorithm and matting algorithm for image preprocessing, the exact position of non-common saliency objects (called Region of Interest here, i.e. ROI) in the image group can be obtained. In the attention retargeting algorithm, the internal features of the image to adjust the saliency of the ROI areas are considered. In the HSI colour space, the three components H, S, and I are adjusted separately. First, the hue distribution is constructed by the Dirac kernel function, and then the most similar hue distribution to the surrounding environment is selected as the best hue distribution of ROI areas. The S and I components can be set as the contrast difference between ROI areas and surrounding background areas according to the user's demands. Experimental results show that this method effectively reduces the ROI areas' attraction to the user's visual attention. Moreover, comparing this method with other methods, the saliency adjustment effect achieved is much better, and the processed image is more natural. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

show abstract

Robust Deep Co-Saliency Detection With Group Semantic and Pyramid Attention

Cited by 34 publications

References 45 publications

Denoised Non-Local Neural Network for Semantic Segmentation

Denoised Non-Local Neural Network for Semantic Segmentation

Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering

Consistent image processing based on co‐saliency

Contact Info

Product

Resources

About