Do we look at lights? Using mixture modelling to distinguish between low- and high-level factors in natural image viewing

Vincent, Ben; Baddeley, Roland; Correani, Alessia; Trościanko, Tom; Leonards, Ute

doi:10.1080/13506280902916691

Cited by 62 publications

(57 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The probabilistic framework we use in this paper (10,19) is easily extended to study spatiotemporal effects, by modeling the conditional probability of a fixation given previous fixations (Materials and Methods and ref. 12).…”

Section: Discussionmentioning

confidence: 99%

Information-theoretic model comparison unifies saliency metrics

Kümmerer

Wallis

Bethge

2015

Proc. Natl. Acad. Sci. U.S.A.

147

167

View full text Add to dashboard Cite

Learning the properties of an image associated with human gaze placement is important both for understanding how biological systems explore the environment and for computer vision applications. There is a large literature on quantitative eye movement models that seeks to predict fixations from images (sometimes termed "saliency" prediction). A major problem known to the field is that existing model comparison metrics give inconsistent results, causing confusion. We argue that the primary reason for these inconsistencies is because different metrics and models use different definitions of what a "saliency map" entails. For example, some metrics expect a model to account for image-independent central fixation bias whereas others will penalize a model that does. Here we bring saliency evaluation into the domain of information by framing fixation prediction models probabilistically and calculating information gain. We jointly optimize the scale, the center bias, and spatial blurring of all models within this framework. Evaluating existing metrics on these rephrased models produces almost perfect agreement in model rankings across the metrics. Model performance is separated from center bias and spatial blurring, avoiding the confounding of these factors in model comparison. We additionally provide a method to show where and how models fail to capture information in the fixations on the pixel level. These methods are readily extended to spatiotemporal models of fixation scanpaths, and we provide a software package to facilitate their use.visual attention | eye movements | probabilistic modeling | likelihood | point processes H umans move their eyes about three times/s when exploring the environment, fixating areas of interest with the highresolution fovea. How do we determine where to fixate to learn about the scene in front of us? This question has been studied extensively from the perspective of "bottom-up" attentional guidance (1), often in a "free-viewing" task in which a human observer explores a static image for some seconds while his or her eye positions are recorded (Fig. 1A). Eye movement prediction is also applied in domains from advertising to efficient object recognition. In computer vision the problem of predicting fixations from images is often referred to as "saliency prediction," while to others "saliency" refers explicitly to some set of low-level image features (such as edges or contrast). In this paper we are concerned with predicting fixations from images, taking no position on whether the features that guide eye movements are "low" or "high" level.The field of eye movement prediction is quite mature: Beginning with the influential model of Itti et al. (1), there are now over 50 quantitative fixation prediction models, including around 10 models that seek to incorporate "top-down" effects (see refs. 2-4 for recent reviews and analyses of this extensive literature). Many of these models are designed to be biologically plausible whereas others aim purely at prediction (e.g., ref. 5). Progress is meas...

show abstract

Section: Discussionmentioning

confidence: 99%

Information-theoretic model comparison unifies saliency metrics

Kümmerer

Wallis

Bethge

2015

Proc. Natl. Acad. Sci. U.S.A.

147

167

View full text Add to dashboard Cite

show abstract

“…As visual features such as luminance, contrast, and edge density in scenes have been shown to correlate with the rated semantic informativeness of a location [8], gaze may be guided toward an object due to its cognitive relevance, and the corresponding peak in feature values around foveation may simply be a by-product of this correlation. For example, a recent study by Vincent et al [108] used mixture modeling to isolate the independent contributions of low and high-level visual features to fixations in static scenes. The authors found that low-level features in isolation contributed very little to fixation location compared to the much larger contribution of central bias and preference for foreground objects.…”

Section: Bottom-up Versus Top-down Control During Ongoing Dynamic Scenesmentioning

confidence: 99%

Clustering of Gaze During Dynamic Scene Viewing is Predicted by Motion

et al. 2010

View full text Add to dashboard Cite

Where does one attend when viewing dynamic scenes? Research into the factors influencing gaze location during static scene viewing have reported that low-level visual features contribute very little to gaze location especially when opposed by high-level factors such as viewing task. However, the inclusion of transient features such as motion in dynamic scenes may result in a greater influence of visual features on gaze allocation and coordination of gaze across viewers. In the present study, we investigated the contribution of low-to mid-level visual features to gaze location during free-viewing of a large dataset of videos ranging in content and length. Signal detection analysis on visual features and Gaussian Mixture Models for clustering gaze was used to identify the contribution of visual features to gaze location. The results show that mid-level visual features including corners and orientations can distinguish between actual gaze locations and a randomly sampled baseline. However, temporal features such as flicker, motion, and their respective contrasts were the most predictive of gaze location. Additionally, moments in which all viewers' gaze tightly clustered in the same location could be predicted by motion. Motion and mid-level visual features may influence gaze allocation in dynamic scenes, but it is currently unclear whether this influence is involuntary or due to correlations with higher order factors such as scene semantics.

show abstract

“…They include both bottom-up and top-down factors. This approach was first proposed by Vincent et al [32] and later by Couronné et al [33]. In Vincent's study, the a priori top-down guiding factors are set up according to the semantic of the scenes.…”

Section: Statistical Modelmentioning

confidence: 99%

A Functional and Statistical Bottom-Up Saliency Model to Reveal the Relative Contributions of Low-Level Visual Guiding Factors

2010

View full text Add to dashboard Cite

Springer New York. ISSN : 1866-9956International audienceWhen looking at a scene, we frequently move our eyes to place consecutive interesting regions on the fovea, the retina centre. At each fixation, only this specific foveal region is analyzed in detail by the visual system. The visual attention mechanisms control eye movements and depend on two types of factor: bottom-up and top-down factors. Bottom-up factors include different visual features such as colour, luminance, edges, orientations, etc. In this paper, we evaluate quantitatively the relative contribution of basic low-level features as candidate guiding factors to visual attention and hence to eye movements. We also study how these visual features can be combined in a bottom-up saliency model. Our work consists of three interactive parts: a functional saliency model, a statistical model and eye movement data recorded during free viewing of natural scenes. The functional saliency model, inspired by the primate visual system, decomposes a visual scene into different feature maps. The statistical model indicates which features best explain the recorded eye movements. We show an essential role of high frequency luminance and an important contribution of central fixation bias. The relative contribution of features, calculated by the statistical model, is then used to combine the different feature maps into a saliency map. Finally, the comparison between the saliency model and experimental data confirmed the influence of these contributions

show abstract

Do we look at lights? Using mixture modelling to distinguish between low- and high-level factors in natural image viewing

Cited by 62 publications

References 44 publications

Information-theoretic model comparison unifies saliency metrics

Information-theoretic model comparison unifies saliency metrics

Clustering of Gaze During Dynamic Scene Viewing is Predicted by Motion

A Functional and Statistical Bottom-Up Saliency Model to Reveal the Relative Contributions of Low-Level Visual Guiding Factors

Contact Info

Product

Resources

About