Modeling global scene factors in attention

Torralba, Antonio

doi:10.1364/josaa.20.001407

Cited by 223 publications

(195 citation statements)

References 47 publications

Supporting

Mentioning

188

Contrasting

Unclassified

Order By: Relevance

“…This also confirms the result from the search task used by Underwood et al (2006), in which conspicuity was also ineffective in guiding eye movements. The cognitive override of visual saliency is also a feature of Torralba's (2003) model of contextual cueing, in which the visual context becomes available sufficiently early to allow modulation of the saliency map. The bottom-up saliency map models do not give a good account of scene inspection when inspection can focus upon the detection of a specific and well-defined object.…”

Section: Discussionmentioning

confidence: 99%

Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Underwood

Foulsham

2006

Quarterly Journal of Experimental Psychology

221

182

View full text Add to dashboard Cite

Models of low-level saliency predict that when we first look at a photograph our first few eye movements should be made towards visually conspicuous objects. Two experiments investigated this prediction by recording eye fixations while viewers inspected pictures of room interiors that contained objects with known saliency characteristics. Highly salient objects did attract fixations earlier than less conspicuous objects, but only in a task requiring general encoding of the whole picture. When they were required to detect the presence of a small target, then the visual saliency of non-target objects did not influence fixations. These results support modifications of the model that take the cognitive override of saliency into account by allowing task demands to reduce the saliency weights of task-irrelevant objects.The pictures sometimes contained incongruent objects that were taken from other rooms. These objects were used to test the hypothesis that previous reports of the early fixation of congruent objects have not been consistent because the effect depends upon the visual conspicuity of the incongruent object. There was an effect of incongruency in both experiments, with earlier fixation of objects that violated the gist of the scene, but the effect was only apparent for inconspicuous objects, arguing against this hypothesis. Address for correspondence:Geoffrey Underwood, School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK. Email: geoff.underwood@nottingham.ac.uk or lpywtf@psychology.nottingham.ac.uk We are very grateful to Laurent Itti who generously made his saliency software available to us, and to Editha van Loon who helped install the software. We also thank Mike Burton, John Findlay and two anonymous reviewers for their helpful comments on a previous draft of this paper.Visual saliency and semantic incongruency 2 What attracts our attention when we first look at a picture of a scene such as a kitchen, a football match, or a harbour? Conspicuous objects might be expected to gain early inspection, and in two experiments here we investigated the effects of visual saliency and scene incongruency. The effects of visual and semantic conspicuity were observed in a free inspection task in which viewers prepared for a recognition memory test, and in a search task, in which they looked for a specific object. In each case we asked whether their early eye fixations would be taken to objects that were visually prominent by virtue of characteristics such as brightness and colour, and to objects that violated the gist of the scene by virtue of not being in their expected environment.Itti and Koch (2000) have developed an algorithm that enables the measurement of the visual saliency of an image on the basis of its physical properties, by the identification of peaks in the distribution of intensity and changes in colour and orientation. The algorithm builds an overall "saliency map" of the image that was suggested by Koch and Ullman (1985) to drive attentional selection of regions of displa...

show abstract

Section: Discussionmentioning

confidence: 99%

Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Underwood

Foulsham

2006

Quarterly Journal of Experimental Psychology

221

182

View full text Add to dashboard Cite

show abstract

“…Predictive performance of the original bottom-up saliency map model reduces or vanishes in search tasks (Einhäuser, Rutishauser, & Koch, 2008;Henderson et al, 2007), but inclusion of contextual or task-dependent information can improve the predictions of saliency map algorithms (Navalpakkam & Itti, 2005;Oliva, Torralba, Castelhano, & Henderson, 2003;Torralba, 2003). For evaluating the performance of saliency-map-type models in predicting search in natural scenes, the intuitive strategy of fixating the point of highest saliency is usually suboptimal; instead, the discriminability between target and distractor on the basis of the full map should be utilized (Gao, Mahadevan, & Vasconcelos, 2008;Vincent, Troscianko, & Gilchrist, 2007).…”

Section: Discussionmentioning

confidence: 99%

Saliency on a natural scene background: Effects of color and luminance contrast add linearly

Engmann

Hart

Sieren

et al. 2009

Attention, Perception, & Psychophysics

View full text Add to dashboard Cite

“…Attentional mechanisms such as image saliency and contextual modulation emerge as a natural consequence from such a model (Torralba, 2003a(Torralba, , 2003b.…”

Section: Model Of Object Search and Contextual Guidancementioning

confidence: 99%

Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search.

Torralba

Oliva

Castelhano³

et al. 2006

Psychological Review

Self Cite

1,458

1,361

View full text Add to dashboard Cite

Many experiments have shown that the human visual system makes extensive use of contextual information for facilitating object search in natural scenes. However, the question of how to formally model contextual influences is still open. On the basis of a Bayesian framework, the authors present an original approach of attentional guidance by global scene context. The model comprises 2 parallel pathways; one pathway computes local features (saliency) and the other computes global (scenecentered) features. The contextual guidance model of attention combines bottom-up saliency, scene context, and top-down mechanisms at an early stage of visual processing and predicts the image regions likely to be fixated by human observers performing natural search tasks in real-world scenes.Keywords: eye movements, visual search, context, global feature, Bayesian model According to feature-integration theory (Treisman & Gelade, 1980), the search for objects requires slow serial scanning because attention is necessary to integrate low-level features into single objects. Current computational models of visual attention based on saliency maps have been inspired by this approach, as it allows a simple and direct implementation of bottom-up attentional mechanisms that are not task specific. Computational models of image saliency (Itti, Koch, & Niebur, 1998;Koch & Ullman, 1985;Parkhurst, Law, & Niebur, 2002;Rosenholtz, 1999) provide some predictions about which regions are likely to attract observers' attention. These models work best in situations in which the image itself provides little semantic information and in which no specific task is driving the observer's exploration. In real-world images, the semantic content of the scene, the co-occurrence of objects, and task constraints have been shown to play a key role in modulating where attention and eye movements go

show abstract

Modeling global scene factors in attention

Cited by 223 publications

References 47 publications

Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Saliency on a natural scene background: Effects of color and luminance contrast add linearly

Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search.

Contact Info

Product

Resources

About