Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Abstract

Models of low-level saliency predict that when we first look at a photograph our first few eye movements should be made towards visually conspicuous objects. Two experiments investigated this prediction by recording eye fixations while viewers inspected pictures of room interiors that contained objects with known saliency characteristics. Highly salient objects did attract fixations earlier than less conspicuous objects, but only in a task requiring general encoding of the whole picture. When they were required to detect the presence of a small target, then the visual saliency of non-target objects did not influence fixations. These results support modifications of the model that take the cognitive override of saliency into account by allowing task demands to reduce the saliency weights of task-irrelevant objects.The pictures sometimes contained incongruent objects that were taken from other rooms. These objects were used to test the hypothesis that previous reports of the early fixation of congruent objects have not been consistent because the effect depends upon the visual conspicuity of the incongruent object. There was an effect of incongruency in both experiments, with earlier fixation of objects that violated the gist of the scene, but the effect was only apparent for inconspicuous objects, arguing against this hypothesis. Address for correspondence:Geoffrey Underwood, School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK. Email: geoff.underwood@nottingham.ac.uk or lpywtf@psychology.nottingham.ac.uk We are very grateful to Laurent Itti who generously made his saliency software available to us, and to Editha van Loon who helped install the software. We also thank Mike Burton, John Findlay and two anonymous reviewers for their helpful comments on a previous draft of this paper.Visual saliency and semantic incongruency 2 What attracts our attention when we first look at a picture of a scene such as a kitchen, a football match, or a harbour? Conspicuous objects might be expected to gain early inspection, and in two experiments here we investigated the effects of visual saliency and scene incongruency. The effects of visual and semantic conspicuity were observed in a free inspection task in which viewers prepared for a recognition memory test, and in a search task, in which they looked for a specific object. In each case we asked whether their early eye fixations would be taken to objects that were visually prominent by virtue of characteristics such as brightness and colour, and to objects that violated the gist of the scene by virtue of not being in their expected environment.Itti and Koch (2000) have developed an algorithm that enables the measurement of the visual saliency of an image on the basis of its physical properties, by the identification of peaks in the distribution of intensity and changes in colour and orientation. The algorithm builds an overall "saliency map" of the image that was suggested by Koch and Ullman (1985) to drive attentional selection of regions of displa...

Integrating mechanisms of visual guidance in naturalistic language production

“…This definition of perceptual guidance is compatible with that of visual saliency (Itti & Koch, 2000;Parkhursta, Lawb, & Niebur, 2002). Saliency, as a composite measure of low-level visual information, is positively correlated with the presence of objects (Elazary & Itti, 2008), and found to guide attention in tasks where targets are underspecified (e.g., memorization, Underwood & Foulsham, 2006), and also utilized during situated language production (Gleitman et al, 2007). In our view, the most appropriate way to define visual saliency is in terms of visual clutter, calculated by integrating low-level visual information, e.g., color, with edge information (Rosenholtz, Mansfield, & Jin, 2005).…”

Do Faces Capture the Attention of Individuals with Williams Syndrome or Autism? Evidence from Tracking Eye Movements

“…Half the images were then manipulated to add an 'incongruent' object (a small face) to them. The term 'incongruent' refers to the fact that the item is inserted in the image in a manner in which it would not typically occur (see Underwood & Foulsham, 2005). This is made clearer in Figure 1a.…”

Eyes wide shut: implied social presence, eye tracking and attention

“…For example, researchers interested in attentional selection record where individuals look in scenes (e.g., Itti & Koch, 2000;Underwood & Foulsham, 2006). In addition, to better understand social attention, researchers have focused on biases in looking at social stimuli (e.g., Birmingham et al, 2008) and the characteristics of individual's looking behavior while engaged in social tasks (e.g., conversation; Keysar, Barr, Balin, & Brauner, 2000).…”