A Differential Approach for Gaze Estimation

Abstract

Most non-invasive gaze estimation methods regress gaze directions directly from a single face or eye image. However, due to important variabilities in eye shapes and inner eye structures amongst individuals, universal models obtain limited accuracies and their output usually exhibit high variance as well as subject dependent biases. Thus, increasing accuracy is usually done through calibration, allowing gaze predictions for a subject to be mapped to her actual gaze. In this paper, we introduce a novel approach, which works by directly training a differential convolutional neural network to predict gaze differences between two eye input images of the same subject. Then, given a set of subject specific calibration images, we can use the inferred differences to predict the gaze direction of a novel eye sample. The assumption is that by comparing eye images of the same user, annoyance factors (alignment, eyelid closing, illumination perturbations) which usually plague single image prediction methods can be much reduced, allowing better prediction altogether. Furthermore, the differential network itself can be adapted via finetuning to make predictions consistent with the available user reference pairs. Experiments on 3 public datasets validate our approach which constantly outperforms state-of-the-art methods even when using only one calibration sample or those relying on subject specific gaze adaptation.

Few-Shot Adaptive Gaze Estimation

“…Our re-implementation yields 3.53 • for their method at k = 9 on MPIIGaze, a 1.2 • improvement despite dataset differences. We show statistically significant improvements to [26] across all ranges of k in our MPIIGaze evaluations, with our method only requiring 4 calibration samples to compete with their best performance at k = 256 (see the red and green curves in Fig. 6).…”

“…Few-shot personalization of CNN models in the context of gaze estimation for very low k is very challenging. Two recent approaches [55,26] are the most relevant in this direction, and we provide evaluations on highly competitive re-implementations. Our results are presented in Fig.…”

“…Ours vs Differential Gaze Estimation. Liu et al [26] introduce a CNN architecture for learning to estimate the differences in the gaze yaw and pitch values between pairs of images of the same subject. That is, in order to estimate the gaze their network always requires one reference image of a subject with known gaze values.…”

“…So far Liu et al [26] report the best known accuracy of 4.67 • with 9 calibration samples on MPIIGaze with their differential network architecture. They use the within MPI-IGaze leave-one-subject out evaluation protocol for their experiments.…”

Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis

“…Gaze Adaptation. However, when testing on unknown person, the different personal eye structures such as eye shapes and visual axis limit the performance of both GBMs and ABMs [16]. Some straightforward solutions to this problem have been proposed, such as to learn person-specific models [15,29,43], fine-tune a pre-trained model [19], learn a SVR using a few samples for calibration [14] or learn a differential gaze model [16].…”