If Visual Saliency Predicts Search, Then Why? Evidence from Normal and Gaze-Contingent Search Tasks in Natural Scenes

Abstract

The Itti and Koch (Vision Research 40: 1489-1506, 2000 saliency map model has inspired a wealth of research testing the claim that bottom-up saliency determines the placement of eye fixations in natural scenes. Although saliency seems to correlate with (although not necessarily cause) fixation in free-viewing or encoding tasks, it has been suggested that visual saliency can be overridden in a search task, with saccades being planned on the basis of target features, rather than being captured by saliency. Here, we find that target regions of a scene that are salient according to this model are found quicker than control regions (Experiment 1). However, this does not seem to be altered by filtering features in the periphery using a gaze-contingent display (Experiment 2), and a deeper analysis of the eye movements made suggests that the saliency effect is instead due to the meaning of the scene regions. Experiment 3 supports this interpretation, showing that scene inversion reduces the saliency effect. These results suggest that saliency effects on search may have nothing to do with bottom-up saccade guidance.

Scrambled eyes? Disrupting scene structure impedes focal processing and increases bottom-up guidance

“…Our results add to what has already been learned about the sorts of contextual cues that configural or gist information might provide for search in natural scenes (Foulsham & Underwood, 2011;Henderson, 2003;Vo & Henderson, 2010). Building this contextual information (e.g.…”

“…Previous work has demonstrated that the effects of saliency on search are minimal (Chen & Zelinsky, 2006;Foulsham & Underwood, 2007), and the implication of this research is that when we search, what we know about the target and the scene overrides any bias to look at conspicuous items. Where saliency is predictive of search, it is likely because saliency coincides with other properties of the target that make it easier to find, such as ease with which its location can be predicted in a scene (Foulsham & Underwood, 2011). In the present research, we found that there was almost no correlation between saliency and the time taken to move the eyes to a target in normal scenes.…”

“…The fixation durations and saccade amplitudes were also within the range predicted by previous search tasks in scenes. Foulsham and Underwood (2011) used a search task for regions, in intact scenes without a grid, and found similar RTs and eye movement statistics. Probing memory for a patch has also been used previously (Velisavljevic & Elder, 2008), and our results from Experiment 2 add to what is known about this task.…”

“…This result is consistent with research from experiments that show effects of peripheral information on fixation durations. For example, Loschky and McConkie (2002);van Diepen and d'Ydewalle (2003); Foulsham and Underwood (2011); used a gaze contingent display to mask or blur the information outside a window at fixation, and found that this lengthened average fixation duration. Why did disrupting the global organization of a scene by scrambling shorten fixation durations, when regions tend to be more easily recognized if context is intact?…”

On Findings of Category and Other Concept Cells in the Brain: Some Theoretical Perspectives on Mental Representation

“…Now, imagine a person looking at an advertisement for a zoo at a distant billboard that shows a tiger, a lion, a giraffe, a rhinoceros, and an elephant, all in close proximity to each other and within the person's span of attention [20][21][22]. A distributed representation system would have to process each animal separately, one after the other, in order to recognize them.…”