Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency

Chong, Eunji; Ruiz, Nataniel; Wang, Yongxin; Zhang, Yun; Rozga, Agata; Rehg, James M.

doi:10.1007/978-3-030-01228-1_24

Cited by 114 publications

(54 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important contribution has been provided by the work in [ 96 ]. The proposed model leverages a multi-task learning framework and it takes three inputs: the entire image, the subject’s cropped face, and the location of the face of the subject whose attention must be estimated.…”

Section: Gaze Tracking By Scene Analysismentioning

confidence: 99%

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

Cazzato

Leo

Distante

et al. 2020

Sensors

View full text Add to dashboard Cite

The automatic detection of eye positions, their temporal consistency, and their mapping into a line of sight in the real world (to find where a person is looking at) is reported in the scientific literature as gaze tracking. This has become a very hot topic in the field of computer vision during the last decades, with a surprising and continuously growing number of application fields. A very long journey has been made from the first pioneering works, and this continuous search for more accurate solutions process has been further boosted in the last decade when deep neural networks have revolutionized the whole machine learning area, and gaze tracking as well. In this arena, it is being increasingly useful to find guidance through survey/review articles collecting most relevant works and putting clear pros and cons of existing techniques, also by introducing a precise taxonomy. This kind of manuscripts allows researchers and technicians to choose the better way to move towards their application or scientific goals. In the literature, there exist holistic and specifically technological survey documents (even if not updated), but, unfortunately, there is not an overview discussing how the great advancements in computer vision have impacted gaze tracking. Thus, this work represents an attempt to fill this gap, also introducing a wider point of view that brings to a new taxonomy (extending the consolidated ones) by considering gaze tracking as a more exhaustive task that aims at estimating gaze target from different perspectives: from the eye of the beholder (first-person view), from an external camera framing the beholder’s, from a third-person view looking at the scene where the beholder is placed in, and from an external view independent from the beholder.

show abstract

Section: Gaze Tracking By Scene Analysismentioning

confidence: 99%

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

Cazzato

Leo

Distante

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Their method is based on the pupil center corneal reflection (PCCR) which requires the use of an infrared camera in order to locate the pupil position. Many other works [36,37,12,41,8] uses the saliency information of an image to determine if a person in the same image is looking at a salient object. Our work differs from them as we use the saliency information of an out-of-frame target for precise gaze estimation.…”

Section: Related Workmentioning

confidence: 99%

SalGaze: Personalizing Gaze Estimation using Visual Saliency

Chang

Martino

Qiu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Traditional gaze estimation methods typically require explicit user calibration to achieve high accuracy. This process is cumbersome and recalibration is often required when there are changes in factors such as illumination and pose. To address this challenge, we introduce SalGaze, a framework that utilizes saliency information in the visual content to transparently adapt the gaze estimation algorithm to the user without explicit user calibration. We design an algorithm to transform a saliency map into a differentiable loss map that can be used for the optimization of CNN-based models. SalGaze is also able to greatly augment standard point calibration data with implicit video saliency calibration data using a unified framework. We show accuracy improvements over 24% using our technique on existing methods.

show abstract

“…Their method still relies on a saliency model. Recently, [22] uses a similar combination of gaze and saliency but is also able to predict whether the object of attention lies within the image or not. Finally, [3], [4] merge the problems of saliency and gaze following in the context of human-robot interaction.…”

Section: Related Workmentioning

confidence: 99%

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

Mâsse¹,

Lathuilière²,

Mesejo³

et al. 2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

In this paper we address the problems of detecting objects of interest in a video and of estimating their locations, solely from the gaze directions of people present in the video. Objects can be indistinctly located inside or outside the camera field of view. We refer to this problem as extended gaze following. The contributions of the paper are the followings. First, we propose a novel spatial representation of the gaze directions adopting a top-view perspective. Second, we develop several convolutional encoder/decoder networks to predict object locations and compare them with heuristics and with classical learning-based approaches. Third, in order to train the proposed models, we generate a very large number of synthetic scenarios employing a probabilistic formulation. Finally, our methodology is empirically validated using a publicly available dataset.

show abstract

Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency

Cited by 114 publications

References 32 publications

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

SalGaze: Personalizing Gaze Estimation using Visual Saliency

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

Contact Info

Product

Resources

About