Learning to predict where humans look

Abstract

For many applications in graphics, design, and human computer interaction, it is essential to understand where humans look in a scene. Where eye tracking devices are not a viable option, models of saliency can be used to predict fixation locations. Most saliency approaches are based on bottom-up computation that does not consider top-down image semantics and often does not match actual eye movements. To address this problem, we collected eye tracking data of 15 viewers on 1003 images and use this database as training and testing examples to learn a model of saliency based on low, middle and high-level image features. This large database of eye tracking data is publicly available with this paper.

Fusing generic objectness and visual saliency for salient object detection

“…For vision research, the attention process concerns particularly with two important concepts, namely, saliency and objectness [3,12]. The main focuses of this paper are to investigate the close relationship between the two concepts, introduce a new formulation to effectively approximate them, and propose a useful algorithm for salient object detection.…”

“…To model where we place our eyes in a visual scene, the saliency-based approaches, e.g., [5,10,12,20], are considered to be the mainstream in the vision community. In such techniques, the visual saliency is typically computed in a bottom-up fashion, as is in the computational model by Itti and Koch [10] where information gathering from local image characteristics such as color, intensity, and orientation is used.…”

“…Despite the close connection between saliency and objectness, existing approaches [3,5,12,14,15,19,20] still lack a good computational model to thoroughly explore the two concepts and their mutual effects. Only one single direction of the interactions is considered: The techniques in [5,12,15] exploit object detection to help saliency estimations, while those in [3,14,19,20] use saliency information Figure 2.…”

“…Only one single direction of the interactions is considered: The techniques in [5,12,15] exploit object detection to help saliency estimations, while those in [3,14,19,20] use saliency information Figure 2. Inefficiency of saliency detection and object detection.…”

“…Inefficiency of saliency detection and object detection. From left to right: (a) the original image, (b) the ground-truth saliency by recording eye fixations [12], (c) saliency map detected by [5], (d) region-based saliency map, and (e) the objects detected by [3].…”