Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis

Yu, Yu; Liu, Gang; Odobez, Jean-Marc

doi:10.1109/cvpr.2019.01221

Cited by 96 publications

(69 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is obtained at the cost of a higher memory footprint and computational complexity. It makes the model competitive with respect to the state of the art: for instance the adaptation method in [44] reported an error of 4.2 • on the MPIIGaze dataset, compared to 3.8 • in our case. The Diff-VGG results are similar to those of Diff-NN-Ad (except on MPIIGaze where it works much better).…”

Section: Resultsmentioning

confidence: 60%

A Differential Approach for Gaze Estimation

Liu

Mora

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Most non-invasive gaze estimation methods regress gaze directions directly from a single face or eye image. However, due to important variabilities in eye shapes and inner eye structures amongst individuals, universal models obtain limited accuracies and their output usually exhibit high variance as well as subject dependent biases. Thus, increasing accuracy is usually done through calibration, allowing gaze predictions for a subject to be mapped to her actual gaze. In this paper, we introduce a novel approach, which works by directly training a differential convolutional neural network to predict gaze differences between two eye input images of the same subject. Then, given a set of subject specific calibration images, we can use the inferred differences to predict the gaze direction of a novel eye sample. The assumption is that by comparing eye images of the same user, annoyance factors (alignment, eyelid closing, illumination perturbations) which usually plague single image prediction methods can be much reduced, allowing better prediction altogether. Furthermore, the differential network itself can be adapted via finetuning to make predictions consistent with the available user reference pairs. Experiments on 3 public datasets validate our approach which constantly outperforms state-of-the-art methods even when using only one calibration sample or those relying on subject specific gaze adaptation.

show abstract

Section: Resultsmentioning

confidence: 60%

A Differential Approach for Gaze Estimation

Liu

Mora

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Choi et al [ 18 ] used heterogeneous GANs and CNN models depending on whether people in the images are wearing glasses. For the user-specific pupil adaptation, Yu et al [ 19 ] generated additional training samples through the synthesis of gaze-redirected eye images from existing reference samples. Similar to [ 19 , 20 ] also proposed a framework for a few-shot adaptive gaze estimation for the learning of person-specific gaze networks by applying very few calibration samples.…”

Section: Related Workmentioning

confidence: 99%

Energy Efficient Pupil Tracking Based on Rule Distillation of Cascade Regression Forest

Kim

Jeong

2020

Sensors

View full text Add to dashboard Cite

As the demand for human-friendly computing increases, research on pupil tracking to facilitate human–machine interactions (HCIs) is being actively conducted. Several successful pupil tracking approaches have been developed using images and a deep neural network (DNN). However, common DNN-based methods not only require tremendous computing power and energy consumption for learning and prediction; they also have a demerit in that an interpretation is impossible because a black-box model with an unknown prediction process is applied. In this study, we propose a lightweight pupil tracking algorithm for on-device machine learning (ML) using a fast and accurate cascade deep regression forest (RF) instead of a DNN. Pupil estimation is applied in a coarse-to-fine manner in a layer-by-layer RF structure, and each RF is simplified using the proposed rule distillation algorithm for removing unimportant rules constituting the RF. The goal of the proposed algorithm is to produce a more transparent and adoptable model for application to on-device ML systems, while maintaining a precise pupil tracking performance. Our proposed method experimentally achieves an outstanding speed, a reduction in the number of parameters, and a better pupil tracking performance compared to several other state-of-the-art methods using only a CPU.

show abstract

“…Gaze Estimation. In the past few years, gaze estimation draws increasing attention because it provides a great way for human-machine interaction [28,24,30]. Appearance based methods achieve promising results through using deep convolutional neural network (CNN).…”

Section: Related Workmentioning

confidence: 99%

A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone

Guo

Liu

Zhang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Gaze estimation for ordinary smart phone, e.g. estimating where the user is looking at on the phone screen, can be applied in various applications. However, the widely used appearance-based CNN methods still have two issues for practical adoption. First, due to the limited dataset, gaze estimation is very likely to suffer from over-fitting, leading to poor accuracy at run time. Second, the current methods are usually not robust, i.e. their prediction results having notable jitters even when the user is performing gaze fixation, which degrades user experience greatly. For the first issue, we propose a new tolerant and talented (TAT) training scheme, which is an iterative random knowledge distillation framework enhanced with cosine similarity pruning and aligned orthogonal initialization. The knowledge distillation is a tolerant teaching process providing diverse and informative supervision. The enhanced pruning and initialization is a talented learning process prompting the network to escape from the local minima and re-born from a better start. For the second issue, we define a new metric to measure the robustness of gaze estimator, and propose an adversarial training based Disturbance with Ordinal loss (DwO) method to improve it. The experimental results show that our TAT method achieves state-of-the-art performance on GazeCapture dataset, and that our DwO method improves the robustness while keeping comparable accuracy. position estimation. 3-D gaze vector estimation is to pre-arXiv:1910.07331v1 [cs.CV]

show abstract

Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis

Cited by 96 publications

References 40 publications

A Differential Approach for Gaze Estimation

A Differential Approach for Gaze Estimation

Energy Efficient Pupil Tracking Based on Rule Distillation of Cascade Regression Forest

A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone

Contact Info

Product

Resources

About