Spatial statistics and attentional dynamics in scene viewing

Engbert, Ralf; Trukenbrod, Hans A.; Barthelmé, Simon; Wichmann, Felix A.

doi:10.1167/15.1.14

Cited by 99 publications

(173 citation statements)

References 41 publications

Supporting

Mentioning

170

Contrasting

Order By: Relevance

“…Second, a probabilistic model allows the examination of any statistical moments of the probability distribution that might be of practical interest. For example, Engbert et al (12) examine the properties of second-order correlations between fixations in scanpaths. Third, information gain allows the contribution of different factors in explaining data variance to be quantified.…”

Section: Discussionmentioning

confidence: 99%

“…The probabilistic framework we use in this paper (10,19) is easily extended to study spatiotemporal effects, by modeling the conditional probability of a fixation given previous fixations (Materials and Methods and ref. 12).…”

Section: Discussionmentioning

confidence: 99%

“…Accounting for the entirety of human eye movement behavior in naturalistic settings will require incorporating information about the task, high-level scene properties, and mechanistic constraints on the eye movement system (12,(15)(16)(17)(20)(21)(22). Our gold standard contains the influence of high-level (but still purely imagedependent) factors to the extent that they are consistent across observers.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Information-theoretic model comparison unifies saliency metrics

Kümmerer

Wallis

Bethge

2015

Proc. Natl. Acad. Sci. U.S.A.

147

167

View full text Add to dashboard Cite

Learning the properties of an image associated with human gaze placement is important both for understanding how biological systems explore the environment and for computer vision applications. There is a large literature on quantitative eye movement models that seeks to predict fixations from images (sometimes termed "saliency" prediction). A major problem known to the field is that existing model comparison metrics give inconsistent results, causing confusion. We argue that the primary reason for these inconsistencies is because different metrics and models use different definitions of what a "saliency map" entails. For example, some metrics expect a model to account for image-independent central fixation bias whereas others will penalize a model that does. Here we bring saliency evaluation into the domain of information by framing fixation prediction models probabilistically and calculating information gain. We jointly optimize the scale, the center bias, and spatial blurring of all models within this framework. Evaluating existing metrics on these rephrased models produces almost perfect agreement in model rankings across the metrics. Model performance is separated from center bias and spatial blurring, avoiding the confounding of these factors in model comparison. We additionally provide a method to show where and how models fail to capture information in the fixations on the pixel level. These methods are readily extended to spatiotemporal models of fixation scanpaths, and we provide a software package to facilitate their use.visual attention | eye movements | probabilistic modeling | likelihood | point processes H umans move their eyes about three times/s when exploring the environment, fixating areas of interest with the highresolution fovea. How do we determine where to fixate to learn about the scene in front of us? This question has been studied extensively from the perspective of "bottom-up" attentional guidance (1), often in a "free-viewing" task in which a human observer explores a static image for some seconds while his or her eye positions are recorded (Fig. 1A). Eye movement prediction is also applied in domains from advertising to efficient object recognition. In computer vision the problem of predicting fixations from images is often referred to as "saliency prediction," while to others "saliency" refers explicitly to some set of low-level image features (such as edges or contrast). In this paper we are concerned with predicting fixations from images, taking no position on whether the features that guide eye movements are "low" or "high" level.The field of eye movement prediction is quite mature: Beginning with the influential model of Itti et al. (1), there are now over 50 quantitative fixation prediction models, including around 10 models that seek to incorporate "top-down" effects (see refs. 2-4 for recent reviews and analyses of this extensive literature). Many of these models are designed to be biologically plausible whereas others aim purely at prediction (e.g., ref. 5). Progress is meas...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Information-theoretic model comparison unifies saliency metrics

Kümmerer

Wallis

Bethge

2015

Proc. Natl. Acad. Sci. U.S.A.

147

167

View full text Add to dashboard Cite

show abstract

“…From these advances, the problem of modeling priority maps seems 19 basically solved [11]: for an arbitrary natural image, computational models can generate 20 a prediction of fixation density in experiments with human observers. 21 The next step in modeling human visual behavior is fundamentally related to the 22 36Hypothesis-based models rely on cognitive and neural assumptions of human 37 April 3, 2020 2/17 perception and oculomotor control that were derived from known biological mechanism 38 and well-established experimental effects [14][15][16][17]. Thus, the key goals of parametric 39 models are (i) to implement these assumptions in a fully quantitative way and build a 40 generative model, (ii) to fit the model to experimental data for hypothesis testing 41 (statistical inference), and, finally, (iii) to provide explanations for interindividual 42 differences in experimental data sets [18].…”

mentioning

confidence: 99%

“…While various 85 models for the computation of static priority maps exist, we extend the modeling approach to the generation of scan paths for a given static saliency map. For simplicity, 87 we use the time-averaged fixation density [16] as an approximation of the saliency of a 88 given image.…”

mentioning

confidence: 99%

A Mathematical Model of Exploration and Exploitation in Natural Scene Viewing

Malem-Shinitski

Opper

Reich

et al. 2020

Preprint

View full text Add to dashboard Cite

Understanding the decision process underlying gaze control is an important question in cognitive neuroscience with applications in diverse fields ranging from psychology to computer vision. The decision for choosing an upcoming saccade target can be framed as a dilemma: Should the observer further exploit the information near the current gaze position or continue with exploration of other patches within the given scene? While several models attempt to describe the dynamics of saccade target selection, none of them explicitly addresses the underlying Exploration-Exploitation dilemma. Here we propose and investigate a mathematical model motivated by the Exploration-Exploitation dilemma in scene viewing. The model is derived from a minimal set of assumptions that generates realistic eye movement behavior. We implemented a Bayesian approach for model parameter inference based on the model's likelihood function. In order to simplify the inference, we applied data augmentation methods that allowed the use of conjugate priors and the construction of an efficient Gibbs sampler. This approach turned out to be numerically efficient and permitted fitting interindividual differences in saccade statistics. Thus, the main contribution of our modeling approach is two-fold; first, we propose a new model for saccade generation in scene viewing. Second, we demonstrate the use of novel methods from Bayesian inference in the field of scan path modeling. Author summaryThe Exploration-Exploitation dilemma is general concept that has been investigated in human information processing. We investigate whether the Exploration-Exploitation trade-off is a viable approach to model sequences of fixations generated by a human observer in a free viewing task with natural scenes. Variants of the basic model are used to predict to the experimental data based on Bayesian inference. Results indicate a high predictive power for both aggregated data and individual differences across observers. The combination of a novel model with state-of-the-art Bayesian methods lends support to the Exploration-Exploitation framework in the field of eye-movement research. Introduction 1 The human visual system acquires high-acuity information from a rather small region 2 (the fovea) surrounding the center of gaze [1]. The foveal organization of the visual 3 April 3, 2020 1/17 system has two immediate consequences. First, visual perception of natural scenes 4 depends critically on the control of precise and fast eye movements (saccades) that move 5 regions of interest into the fovea for high-acuity processing. During a typical visual task 6 (e.g., scene viewing or reading), saccades occur at a rate of 3 to 4 per second [2]. Second, 7 the decision process for an upcoming saccade target poses a dilemma: should the 8 observer further exploit the information near the fovea or continue with exploration of 9 other patches within the given scene? The latter problem is critical for scene 10 viewing [3, 4] and relevant to the broader field of cognitive processes in knowledge ...

show abstract

Methods and Models of Eye-Tracking in Natural Environments

Harston

Faisal

2022

Neuromethods

View full text Add to dashboard Cite

Spatial statistics and attentional dynamics in scene viewing

Cited by 99 publications

References 41 publications

Information-theoretic model comparison unifies saliency metrics

Information-theoretic model comparison unifies saliency metrics

A Mathematical Model of Exploration and Exploitation in Natural Scene Viewing

Methods and Models of Eye-Tracking in Natural Environments

Contact Info

Product

Resources

About