Weakly Supervised Saliency Detection with A Category-Driven Map Generator

Hsu, Ken-Yuh; Lin, Yen‐Yu; Chuang, Yung-Yu

doi:10.5244/c.31.67

Cited by 13 publications

(7 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Having obtained the spatial saliency maps by selecting maximum per-pixel responses, we can then use these spatial heatmaps to learn or predict temporal saliency. More recently, Hsu et al [21] develop two coupled ConvNets, one for image-level classifier and the other for pixel-level generator. By designing a well-formulated loss function and top-down guidance from class labels, the generator is demonstrated to output saliency estimation of good quality.…”

Section: Related Workmentioning

confidence: 99%

“…Temporal consistent loss. Inspired by [21,67,16] that model correlation between discrete images in an selfsupervised manner by per-pixel displacement warping, smoothness regularization, etc., we design 3 loss functions to train our model and refine O t by temporal constraints: temporal reconstruction loss L recons , smoothness loss L smooth , and motion masking loss L motion . The total loss function of each time step t can be formulated as:…”

Section: Temporal Modelmentioning

confidence: 99%

See 1 more Smart Citation

Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos

Cheng

Chao

Dong

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

184

127

View full text Add to dashboard Cite

Automatic saliency prediction in 360 • videos is critical for viewpoint guidance applications (e.g., Facebook 360 Guide). We propose a spatial-temporal network which is (1) weakly-supervised trained and (2) tailor-made for 360 • viewing sphere. Note that most existing methods are less scalable since they rely on annotated saliency map for training. Most importantly, they convert 360 • sphere to 2D images (e.g., a single equirectangular image or multiple separate Normal Field-of-View (NFoV) images) which introduces distortion and image boundaries. In contrast, we propose a simple and effective Cube Padding (CP) technique as follows. Firstly, we render the 360 • view on six faces of a cube using perspective projection. Thus, it introduces very little distortion. Then, we concatenate all six faces while utilizing the connectivity between faces on the cube for image padding (i.e., Cube Padding) in convolution, pooling, convolutional LSTM layers. In this way, CP introduces no image boundary while being applicable to almost all Convolutional Neural Network (CNN) structures. To evaluate our method, we propose Wild-360, a new 360 • video saliency dataset, containing challenging videos with saliency heatmap annotations. In experiments, our method outperforms baseline methods in both speed and quality.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Temporal Modelmentioning

confidence: 99%

Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos

Cheng

Chao

Dong

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

184

127

View full text Add to dashboard Cite

show abstract

“…On the other hand, previous works [15], [40], [59]- [61] have already demonstrated that semantic information, especially Fig. 5: The detailed architecture of the proposed bi-stream network.…”

Section: B Which Training Set Should Be Selected?mentioning

confidence: 96%

A Deeper Look at Salient Object Detection: Bi-stream Network with a Small Training Dataset

Wu,

Li,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Compared with the conventional hand-crafted approaches, the deep learning based methods have achieved tremendous performance improvements by training exquisitely crafted fancy networks over large-scale training sets. However, do we really need large-scale training set for salient object detection (SOD)? In this paper, we provide a deeper insight into the interrelationship between the SOD performances and the training sets. To alleviate the conventional demands for large-scale training data, we provide a feasible way to construct a novel smallscale training set, which only contains 4K images. Moreover, we propose a novel bi-stream network to take full advantage of our proposed small training set, which is consisted of two feature backbones with different structures, achieving complementary semantical saliency fusion via the proposed gate control unit. To our best knowledge, this is the first attempt to use a smallscale training set to outperform state-of-the-art models which are trained on large-scale training sets; nevertheless, our method can still achieve the leading state-of-the-art performance on five benchmark datasets. Both the code and dataset are publicly available at https://github.com/wuzhenyubuaa/TSNet.

show abstract

“…Wang et al [34] introduced a foreground inference network to produce potential saliency maps using image-level labels. Hsu et al [10] presented a categorydriven map generator to learn saliency from image-level labels. Similarly, Li et al [16] adopted an iterative learning strategy to update an initial saliency map generated from unsupervised saliency methods by learning saliency from image-level supervision.…”

Section: Learning Saliency From Weak Annotationsmentioning

confidence: 99%

Weakly-Supervised Salient Object Detection via Scribble Annotations

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Compared with laborious pixel-wise dense labeling, it is much easier to label data by scribbles, which only costs 1∼2 seconds to label one image. However, using scribble labels to learn salient object detection has not been explored. In this paper, we propose a weakly-supervised salient object detection model to learn saliency from such annotations. In doing so, we first relabel an existing largescale salient object detection dataset with scribbles, namely S-DUTS dataset. Since object structure and detail information is not identified by scribbles, directly training with scribble labels will lead to saliency maps of poor boundary localization. To mitigate this problem, we propose an auxiliary edge detection task to localize object edges explicitly, and a gated structure-aware loss to place constraints on the scope of structure to be recovered. Moreover, we design a scribble boosting scheme to iteratively consolidate our scribble annotations, which are then employed as supervision to learn high-quality saliency maps. As existing saliency evaluation metrics neglect to measure structure alignment of the predictions, the saliency map ranking metric may not comply with human perception. We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps, which is more consistent with human perception. Extensive experiments on six benchmark datasets demonstrate that our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models 1 .

show abstract

Weakly Supervised Saliency Detection with A Category-Driven Map Generator

Cited by 13 publications

References 40 publications

Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos

Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos

A Deeper Look at Salient Object Detection: Bi-stream Network with a Small Training Dataset

Weakly-Supervised Salient Object Detection via Scribble Annotations

Contact Info

Product

Resources

About