Predicting visual fixations on video based on low-level visual features

Abstract

To what extent can a computational model of the bottom-up visual attention predict what an observer is looking at? What is the contribution of the low-level visual features in the attention deployment? To answer these questions, a new spatio-temporal computational model is proposed. This model incorporates several visual features; therefore, a fusion algorithm is required to combine the different saliency maps (achromatic, chromatic and temporal). To quantitatively assess the model performances, eye movements were recorded while naive observers viewed natural dynamic scenes. Four completing metrics have been used. In addition, predictions from the proposed model are compared to the predictions from a state of the art model [Itti's model (Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254-1259)] and from three non-biologically plausible models (uniform, flicker and centered models). Regardless of the metric used, the proposed model shows significant improvement over the selected benchmarking models (except the centered model). Conclusions are drawn regarding both the influence of low-level visual features over time and the central bias in an eye tracking experiment.

Fusion of Multiple Visual Cues for Visual Saliency Extraction from Wearable Camera Settings with Strong Motion

“…Our goal is now to extract the local motion in video frames i.e. residual motion with regard to model (4). We denote the macro-block optical flow motion vector V c (I, i).…”

“…Numerous psycho-visual studies which have been conducted since the last quarter of 20th century uncovered some factors influencing it. Considering only signal features, the sensitivity to color contrasts, contours, orientation and motion observed in image plane has been stated by numerous authors [3,4]. Nevertheless, only these features are not sufficient to delimit the area in the image plane which is the strongest gaze attractor.…”

Contextual cropping and scaling of TV productions

“…Complex saliency models like the ones by Itti et al [3] or le Meur [9] are close to the perception of the human visual system. In some cases, however, they may be too general, because they rely on pure bottom-up information.…”

“…The applied visual attention model [9] considers color contrast, visual masking effects and orientation features. Furthermore hierarchical block matching, a 2D affine motion model and M-estimator regression is used to determine temporal saliency.…”

Flexible Actuator With Variable Stiffness and Its Decoupling Control Algorithm: Principle Prototype Design and Experimental Verification

“…The early work is in [25], Itti et al derived saliency map by means of calculating the images' pixel intensity, orientation and color's contrast. Then many improved algorithms emerged in large numbers, like the method based on natural statistics in [26], the algorithm which mainly utilized the low-level features in [27], the approach which considered the spectral residual in [28], etc. Some VQA methods also viewed saliency as a powerful tool.…”