SALICON: Saliency in Context

Jiang, Ming; Huang, Shengsheng; Duan, Juanyong; Zhao, Qi

doi:10.1109/cvpr.2015.7298710

Cited by 595 publications

(568 citation statements)

References 27 publications

Supporting

Mentioning

565

Contrasting

Unclassified

Order By: Relevance

“…This provided a more complete set of annotations than the regions extracted for the MTurk labeling tasks, where only the top 1-3 most highly-fixated regions per image were labeled. In this section we compute the importance of faces in an image following the approach of Jiang et al [18]: given a bounding box for an object in an image, the maximum saliency value falling within the object's outline is taken as the object's importance score (the maximum is a good choice for such analyses as it does not scale with object size).…”

Section: The Importance Of Peoplementioning

confidence: 99%

“…DeepFix [19] and SALICON [18], both neural network models, hold the top 2 spots. The CAT2000 dataset, a recent addition to the MIT benchmark, has 19 models evaluated to date.…”

Section: Evaluating Progressmentioning

confidence: 99%

“…Similarly, Liu et al [17] proposed the Multiresolution-CNN model in which three convnets, each on a different image scale, are combined to obtain the saliency map. In the SALICON model [18], CNNs are applied at two different image scales: fine and coarse. The SALICON dataset, a large-scale crowd-sourced mouse movement dataset, made available to the saliency community for training new deep models [18], has led to the emergence of a number of other neural network models.…”

Section: Related Workmentioning

confidence: 99%

“…In the SALICON model [18], CNNs are applied at two different image scales: fine and coarse. The SALICON dataset, a large-scale crowd-sourced mouse movement dataset, made available to the saliency community for training new deep models [18], has led to the emergence of a number of other neural network models. For instance, DeepFix [19] is a fully convolutional neural network built on top of the VGG network [20] and trained on the SALICON dataset to predict pixel-wise saliency values in an end-to-end manner.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Where Should Saliency Models Look Next?

Bylinskii

Recasens

Borji

et al. 2016

Lecture Notes in Computer Science

171

197

View full text Add to dashboard Cite

Abstract. Recently, large breakthroughs have been observed in saliency modeling. The top scores on saliency benchmarks have become dominated by neural network models of saliency, and some evaluation scores have begun to saturate. Large jumps in performance relative to previous models can be found across datasets, image types, and evaluation metrics. Have saliency models begun to converge on human performance? In this paper, we re-examine the current state-of-the-art using a finegrained analysis on image types, individual images, and image regions. Using experiments to gather annotations for high-density regions of human eye fixations on images in two established saliency datasets, MIT300 and CAT2000, we quantify up to 60% of the remaining errors of saliency models. We argue that to continue to approach human-level performance, saliency models will need to discover higher-level concepts in images: text, objects of gaze and action, locations of motion, and expected locations of people in images. Moreover, they will need to reason about the relative importance of image regions, such as focusing on the most important person in the room or the most informative sign on the road. More accurately tracking performance will require finer-grained evaluations and metrics. Pushing performance further will require higher-level image understanding.

show abstract

Section: The Importance Of Peoplementioning

confidence: 99%

“…DeepFix [19] and SALICON [18], both neural network models, hold the top 2 spots. The CAT2000 dataset, a recent addition to the MIT benchmark, has 19 models evaluated to date.…”

Section: Evaluating Progressmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Where Should Saliency Models Look Next?

Bylinskii

Recasens

Borji

et al. 2016

Lecture Notes in Computer Science

171

197

View full text Add to dashboard Cite

show abstract

“…Note that the red boxes, which do not correspond to objects, let alone salient ones, all have higher scores than the green box, which does denote a salient object. Right the saliency map output by the saliency detection method of Jiang et al (2015), currently the highest ranking method on the MIT saliency benchmark (Bylinskii et al 2012). Note that the cooler is not highlighted as salient.…”

Section: Figmentioning

confidence: 99%

Sentence Directed Video Object Codiscovery

Yu¹,

Siskind

2017

Int J Comput Vis

View full text Add to dashboard Cite

Video object codiscovery can leverage the weak semantic constraint implied by sentences that describe the video content. Our codiscovery method, like other object codetection techniques, does not employ any pretrained object models or detectors. Unlike most prior work that focuses on codetecting large objects which are usually salient both in size and appearance, our method can discover small or medium sized objects as well as ones that may be occluded for part of the video. More importantly, our method can codiscover multiple object instances of different classes within a single video clip. Although the semantic information employed is usually simple and weak, it can greatly boost performance by constraining the hypothesized object locations. Experiments show promising results on three datasets: an average IoU score of 0.423 on a new dataset with 15 object

show abstract

Saliency prediction based on object recognition and gaze analysis

Ishikawa

Yakoh

2021

Elect Comm in Japan

View full text Add to dashboard Cite

Predicting the human visual attention in an image is called saliency prediction and is an active research area in the field of neuroscience and computer vision. Early works on saliency prediction was performed by using low‐level features. In recent years, convolutional neural networks have been adapted for saliency prediction and achieved the state‐of‐the‐art performance. However, the eye‐gaze depends on the personality of each viewer and conventional methods did not take into account such individual properties of the viewer. Therefore, this paper proposes a novel saliency prediction method considering the influence of eye‐gaze. Assuming that personality can be expressed as the degree of attention to an object, our proposed method considers the personality by learning which objects are likely to be perceived by each viewer and weighting the universal saliency map with the generated mask based on the object detection results. The experimental results show that the proposed universal saliency map achieves higher accuracy than conventional methods on the public dataset, and the proposed weighted saliency map can reflect the variation of the eye‐gaze influences among viewers.

show abstract

SALICON: Saliency in Context

Cited by 595 publications

References 27 publications

Where Should Saliency Models Look Next?

Where Should Saliency Models Look Next?

Sentence Directed Video Object Codiscovery

Saliency prediction based on object recognition and gaze analysis

Contact Info

Product

Resources

About