Learning to find eye region landmarks for remote gaze estimation in unconstrained settings

Abstract

Conventional feature-based and model-based gaze estimation methods have proven to perform well in settings with controlled illumination and specialized cameras. In unconstrained real-world settings, however, such methods are surpassed by recent appearance-based methods due to difficulties in modeling factors such as illumination changes and other visual artifacts. We present a novel learningbased method for eye region landmark localization that enables conventional methods to be competitive to latest appearance-based methods. Despite having been trained exclusively on synthetic data, our method exceeds the state of the art for iris localization and eye shape registration on real-world imagery. We then use the detected landmarks as input to iterative model-fitting and lightweight learning-based gaze estimation methods. Our approach outperforms existing model-fitting and appearance-based methods in the context of person-independent and personalized gaze estimation.

Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications

“…The main goal of our experiments was to study the accuracy gap of state-of-the-art appearance-based gaze estimation (represented by MPIIFaceGaze [72]) with a model-based counterpart (represented by GazeML [47]) as well as a commercial eye tracker (Tobii EyeX). The GazeML pupil detector was trained on large-scale synthetic eye images [64] with deep convolutional neural networks.…”

“…In order to achieve this goal, we make the following contributions: First, we evaluate the accuracy of state-of-the-art appearancebased gaze estimation for interaction scenarios with and without personal calibration, indoors and outdoors, for different interaction distances, as well as for users with and without glasses. We compare accuracy with a state-of-theart model-based gaze estimation method [47] and, for the first time, with a commercial eye tracker. Second, we discuss the obtained findings and their implications for the most important gaze-based applications [36] ranging from explicit eye input, to attentive user interfaces and gaze-based user modelling, to passive eye monitoring.…”

Accurate and robust eye center localization via fully convolutional networks

“…Note that all the aforementioned methods are used for semantic segmentation. Recently, however, the FCN-like network structure has been also applied successfully to other keypoint detection problems such as human pose estimation [58], facial landmark detection [59] and eye gaze estimation [60,61]. They all have an encoder-decoder architecture and used a FCN-like network structure called hourglass network which borrows the idea from FCN.…”

SalGaze: Personalizing Gaze Estimation using Visual Saliency

“…Krafka et al [24] used Amazon Turk to collect over 2 million images of people using iPhones and iPads. Park et al [35,34] trained a deep network to regress to intermediate eye landmarks and pictorial representations of the eyeball before estimating the gaze. These contributions have made significant breakthroughs in the area of calibration-free person-independent gaze estimation which shows the importance of having large amounts of data.…”