Empirical Validation of the Saliency-based Model of Visual Attention

Abstract

Visual attention is the ability of the human vision system to detect salient parts of the scene, on which higher vision tasks, such as recognition, can focus. In human vision, it is believed that visual attention is intimately linked to the eye movements and that the fixation points correspond to the location of the salient scene parts. In computer vision, the paradigm of visual attention has been widely investigated and a saliencybased model of visual attention is now available that is commonly accepted and used in the field, despite the fact that its biological grounding has not been fully assessed. This work proposes a new method for quantitatively assessing the plausibility of this model by comparing its performance with human behavior. The basic idea is to compare the map of attention -the saliency map -produced by the computational model with a fixation density map derived from eye movement experiments. This human attention map can be constructed as an integral of single impulses located at the positions of the successive fixation points. The resulting map has the same format as the computer-generated map, and can easily be compared by qualitative and quantitative methods. Some illustrative examples using a set of natural and synthetic color images show the potential of the validation method to assess the plausibility of the attention model.

A Behavioral Analysis of Computational Models of Visual Attention

“…This contrasts with studies which use Euclidean-based measures and is more similar, but not equivalent to, those studies that employ similarity based measures. Second, our work is not compatible with previous works which operate over static images (Ouerhani et al, 2004;Parkhurst et al, 2002 and subsequent discussions). The addition of a temporal component complicates analysis: human scan trajectories cannot be collapsed across the time dimension when the underlying substrate of attention, the visual scene, is time-varying.…”

“…Parkhurst et al (2002) show that the saliency maps of images, as computed by the Itti model, is higher in locations fixated upon by human subjects than would have been expected by chance alone. Ouerhani et al (2004) show that the saliency maps generated by the same computational attention model are correlated to approximate probability density maps of humans. In Itti et al (2003) temporal flicker and a Reichardt model for motion are added to the Itti model, allowing for analysis of dynamic scenes.…”

“…This is the method employed in both Parkhurst et al (2002) and Ouerhani et al (2004). Notably, both groups use the locations that human subjects fixate upon to index into the saliency map, and show that the saliency at the locations attended to by humans is greater than what would be expected by a random process.…”

Computational Models of Human Visual Attention and Their Implementations: A Survey

“…Some studies estimated human attention map resulted from convolving a Gaussian kernel to the gaze-point distributions and measure the correlation between the human attention map and computed saliency maps [67], [112].…”

Model Performance for Visual Attention in Real 3D Color Scenes

“…However, and despite the fact that it is inspired by psychophysical studies, only few works have addressed the biological plausibility of the saliency-based model [8]. Parkhurst et al [9] presented for the first time a quantitative comparison between the computational model and human visual attention.…”