HARF: Hierarchy-Associated Rich Features for Salient Object Detection

Zou, Wenbin; Komodakis, Nikos

doi:10.1109/iccv.2015.54

Cited by 53 publications

(20 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spatiotemporal saliency models. Over the recent decades, a variety of techniques and theories have be exploited to detect salient objects in still images, such as spatial prior [15], lowrank matrix recovery [16], regional contrast [17], graphical modeling [18], and information theory [19].…”

Section: Related Workmentioning

confidence: 99%

Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks

Tang

Zou

Jin

et al. 2019

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

Recently, deep learning techniques have substantially boosted the performance of salient object detection in still images. However, the salient object detection in videos by using traditional handcrafted features or deep learning features is not fully investigated, probably due to the lack of sufficient manually labeled video data for saliency modeling, especially for the data-driven deep learning. This paper proposes a novel weakly supervised approach to salient object detection in a video, which can learn a robust saliency prediction model by using very limited manually labeled data and a large amount of weakly labeled data that could be easily generated in a supervised approach. Furthermore, we propose a spatiotemporal cascade neural network (SCNN) architecture for saliency modeling, in which two fully convolutional networks are cascaded to evaluate visual saliency from both spatial and temporal cues to lead the optimal video saliency prediction. The proposed approach is extensively evaluated on the widely used challenging datasets, and the experiments demonstrate that our proposed approach substantially outperforms the state-of-the-art salient object detection models. Index Terms-Video saliency, weakly supervised learning, spatiotemporal prior fusion, cascade fully convolutional network I. INTRODUCTION S ALIENT object detection, which aims to identify the objects or regions that are noticeable and mostly attract human attention in an image/video, has become a research focus of computer vision for decades. It is generally as a preprocessing step to support high-level computer vision tasks, such as object segmentation, object recognition, object tracking and content-based video compression. A number of approaches have been proposed to detect salient objects. The recent approaches based on deep Convolutional Neural Networks (CNNs), e.g., [1]-[3], have substantially improved

show abstract

Section: Related Workmentioning

confidence: 99%

Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks

Tang

Zou

Jin

et al. 2019

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…supervised and unsupervised approaches. Most supervised methods including those using deep learning [29][30][31][32][33] are able to obtain good saliency maps, where high performance computers even with particular graphic process units (GPU) are needed to cope with the lengthy training time. In addition, supervised methods may also suffer from lack of generality, especially when the training samples are limited and/or insufficiently representative.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement

Yan

Ren

Sun

et al. 2018

Pattern Recognition

143

View full text Add to dashboard Cite

Visual attention is a kind of fundamental cognitive capability that allows human beings to focus on the region of interests (ROIs) under complex natural environments. What kind of ROIs that we pay attention to mainly depends on two distinct types of attentional mechanisms. The bottom-up mechanism can guide our detection of the salient objects and regions by externally driven factors, i.e. color and location, whilst the top-down mechanism controls our biasing attention based on prior knowledge and cognitive strategies being provided by visual cortex. However, how to practically use and fuse both attentional mechanisms for salient object detection has not been sufficiently explored. To the end, we propose in this paper an integrated framework consisting of bottom-up and top-down attention mechanisms that enable attention to be computed at the level of salient objects and/or regions. Within our framework, the model of a bottom-up mechanism is guided by the gestalt-laws of perception. We interpreted gestalt-laws of homogeneity, similarity, proximity and figure and ground in link with color, spatial contrast at the level of regions and objects to produce feature contrast map. The model of top-down mechanism aims to use a formal computational model to describe the background connectivity of the attention and produce the priority map. Integrating both mechanisms and applying to salient object detection, our results have demonstrated that the proposed method consistently outperforms a number of existing unsupervised approaches on five challenging and complicated datasets in terms of higher precision and recall rates, AP (average precision) and AUC (area under curve) values.

show abstract

“…In (Zhao et al, 2015), Zhao et al propose a unified multi-context deep neural network taking both global and local context into consideration. Li et al (Li and Yu, 2015) and Zou et al (Zou and Komodakis, 2015) explore high-quality visual features extracted from DNNs to improve the accuracy of saliency detection. DeepSaliency in (Li et al, 2016) is a multi-task deep neural network using a collaborative feature learning scheme between two correlated tasks, saliency detection and semantic segmentation, to learn better feature representation.…”

Section: Deep Neural Networkmentioning

confidence: 99%

Hierarchical Cellular Automata for Visual Saliency

Qin

Feng

et al. 2018

Int J Comput Vis

View full text Add to dashboard Cite

Saliency detection, finding the most important parts of an image, has become increasingly popular in computer vision. In this paper, we introduce Hierarchical Cellular Automata (HCA) -a temporally evolving model to intelligently detect salient objects. HCA consists of two main components: Single-layer Cellular Automata (SCA) and Cuboid Cellular Automata (CCA). As an unsupervised propagation mechanism, Single-layer Cellular Automata can exploit the intrinsic relevance of similar regions through interactions with neighbors. Low-level image features as well as highlevel semantic information extracted from deep neural networks are incorporated into the SCA to measure the correlation between different image patches. With these hierarchical deep features, an impact factor matrix and a coherence matrix are constructed to balance the influences on each cell's next state. The saliency values of all cells are iteratively updated according to a well-defined update rule. Furthermore, we propose CCA to integrate multiple saliency maps generated by SCA at different scales in a Bayesian framework. Therefore, single-layer propagation and multi-layer integration are jointly modeled in our unified HCA. Surprisingly, we * Equal Contribution find that the SCA can improve all existing methods that we applied it to, resulting in a similar precision level regardless of the original results. The CCA can act as an efficient pixel-wise aggregation algorithm that can integrate state-of-the-art methods, resulting in even better results. Extensive experiments on four challenging datasets demonstrate that the proposed algorithm outperforms state-of-the-art conventional methods and is competitive with deep learning based approaches.

show abstract

HARF: Hierarchy-Associated Rich Features for Salient Object Detection

Cited by 53 publications

References 41 publications

Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks

Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks

Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement

Hierarchical Cellular Automata for Visual Saliency

Contact Info

Product

Resources

About