Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics

Abstract

Abstract-Based on an information theoretical approach, we investigate feature selection processes in saccadic object and scene analysis. Saccadic eye movements of human observers are recorded for a variety of natural and arti cial test images. These experimental data are used for a statistical evaluation of the xated image regions. Analysis of second-order statistics indicates that regions with higher spatial variance have a higher probability to be xated, but no signi cant differences beyond these variance effects could be found at the level of power spectra. By contrast, an investigation with higher-order statistics, as re ected in the bispectral density, yielded clear structural differences between the image regions selected by saccadic eye movements as opposed to regions selected by a random process. These results indicate that nonredundant, intrinsically two-dimensional image features like curved lines and edges, occlusions, isolated spots, etc. play an important role in the saccadic selection process which must be integrated with top-down knowledge to fully predict object and scene analysis by human observers.

Visual saliency does not account for eye movements during visual search in real-world scenes

“…Image statistics of areas selected for eye fixation within scenes differ in systematic ways from areas that are not fixated. A possible interpretation of these results is that fixation position can be accounted for by low-level image statistics (Krieger et al, 2000;Mannan et al, 1995Mannan et al, , 1996Mannan et al, , 1997aParkhust & Niebur, 2003;Reinagel & Zador, 1999). The present results, however, call this interpretation into question.…”

“…For example, high spatial frequency content and edge density have been found to be somewhat greater at fixated than non-fixated locations (Mannan, Ruddock, & Wooding, 1996, 1997b. Furthermore, local contrast (the standard deviation of intensity in a patch) is higher and two-point intensity correlation (intensity of the fixated point and nearby points) is lower for fixated scene patches than control patches (Krieger, Rentschler, Hauske, Schill, & Zetzsche, 2000;Parkhurst & Neibur, 2003;Reinagel & Zador, 1999).…”

“…Several studies have demonstrated that the image properties of fixated regions tend to differ in systematic ways from regions that are not fixated (Krieger et al, 2000;Mannan, et al, 1995Mannan, et al, , 1996Mannan, Ruddock, & Wooding, 1997a;Parkhurst & Niebur, 2003;Reinagel & Zador, 1999). Specifically, fixated scene regions tend to be lower in intensity but higher in edge density and local contrast, and are more likely to contain third-order spatial relationships such as T-junctions and curves, than non-fixated regions.…”

View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds

“…Eye tracking experiments show that where the eyes look is both dependent on the mandates of the task, and the features of the scene (Findlay, 1995, Findlay, 1997, Gilchrist, Heywood & Findlay, 2003. Empty and homogenous locations of the scene are seldom foveated, and lines, borders, and especially corners and intersections attract more fixations (Krieger, Rentschler, Hauske, Schill & Zetzsche, 2000). Given that object corners, intersections, and other singular features are the most informative parts of the scene, and that the fovea represents items with the highest resolution, looking at such singular object features ensures that the brain represents the potentially most informative parts of an object with the highest resolution.…”

Optimal reward harvesting in complex perceptual environments

“…On the other hand, studies on visual saliency did not manipulate the reward outcome associated with choosing an item. These studies showed that during free viewing of natural scenes and videotapes, saccades are automatically drawn to more salient image regions (e.g., locations with high feature-contrast in luminance, orientation, and motion) (9)(10)(11)(12)(13) and that salient targets are detected faster and better amid clutter (1,2). Thus, whether and how the brain might combine information about visual saliency and value to form rapid decisions have not yet been investigated.…”