How is motion integrated into a proto-object based visual saliency model?

Molin, Jamal Lottier; Etienne‐Cummings, Ralph; Niebur, Ernst

doi:10.1109/ciss.2015.7086902

Cited by 11 publications

(22 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A feature map F is computed for each video frame based on bottom-up features. Here, we choose a lowlevel saliency map for F , based on motion, intensity, color and orientation [3], multiplied with a Gaussian anisotropic center bias (as suggested in [4]). The visual sensitivity map S depends on the current gaze position (x 0 , y 0 ) and determines the extent to which features can be perceived by the observer.…”

Section: Eye Movements As a Sequential Decision Making Taskmentioning

confidence: 99%

A modular framework for object-based saccadic decisions in dynamic scenes

Roth,

Bideau,

Hellwich

et al. 2021

Preprint

View full text Add to dashboard Cite

Visually exploring the world around us is not a passive process. Instead, we actively explore the world and acquire visual information over time. Here, we present a new model for simulating human eye-movement behavior in dynamic real-world scenes. We model this active scene-exploration as a sequential decision making process. We adapt the popular drift-diffusion model (DDM) for perceptual decision making and extend it towards multiple options, defined by objects present in the scene. For each possible choice, the model integrates evidence over time and a decision (saccadic eye movement) is triggered as soon as evidence crosses a decision threshold. Drawing this explicit connection between decision making and object-based scene perception is highly relevant in the context of active viewing, where decisions are made continuously while interacting with an external environment. We validate our model with a carefully designed ablation study, and explore influences of our model parameters. A comparison on the VidCom dataset 1 supports the plausibility of the proposed approach.

show abstract

Section: Eye Movements As a Sequential Decision Making Taskmentioning

confidence: 99%

A modular framework for object-based saccadic decisions in dynamic scenes

Roth,

Bideau,

Hellwich

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We do not implement temporal dynamics of neuronal activity because all our stimuli are static images. Other saliency models have been proposed that do include image motion, for instance, Molin et al ( 2015 ).…”

Section: Modelmentioning

confidence: 99%

Proto-Object Based Saliency Model With Texture Detection Channel

Uejima

Niebur

Etienne‐Cummings

2020

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

show abstract

“…A proto-object based saliency model was shown to predict eye fixations with good accuracy (Russell et al, 2014). While originally this model used information from maps of intensity, color, and orientation, it was later extended to additionally utilize motion, depth, and texture features (Hu et al, 2016; Mancinelli et al, 2018; Molin et al, 2015; Uejima et al, 2020), and also implemented in biofidelic neuromorphic hardware (Iacono et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

The influence of stereopsis on visual saliency in a proto-object based model of selective attention

Uejima

Mancinelli

Niebur

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Some animals including humans use stereoscopic vision which reconstructs spatial information about the environment from the disparity between images captured by eyes in two separate adjacent locations. Like other sensory information, such stereoscopic information is expected to influence attentional selection. We develop a biologically plausible model of binocular vision to study its effect on bottom-up visual attention, i.e., visual saliency. In our model, the scene is organized in terms of proto-objects on which attention acts, rather than on unbound sets of elementary features. We show that taking into account the stereoscopic information improves the performance of the model in the prediction of human eye movements with statistically significant differences.

show abstract

How is motion integrated into a proto-object based visual saliency model?

Cited by 11 publications

References 14 publications

A modular framework for object-based saccadic decisions in dynamic scenes

A modular framework for object-based saccadic decisions in dynamic scenes

Proto-Object Based Saliency Model With Texture Detection Channel

The influence of stereopsis on visual saliency in a proto-object based model of selective attention

Contact Info

Product

Resources

About