Gaze Estimation via a Differential Eyes’ Appearances Network with a Reference Grid

Gu, Song; Wang, Lihui; He, Long; He, Xianding; Wang, Jian

doi:10.1016/j.eng.2020.08.027

Cited by 15 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Literature [10] uses dilated convolution to expand the perception area of the model to acquire richer global information. Besides, dilated spatial convolution pooling pyramid [1] [11] is adopted to extract multi-scale global information. Ranftl et al [12] used Transformer to replace convolutional neural networks as feature extractors and applied it to dense prediction problems.…”

Section: Video Depth Estimationmentioning

confidence: 99%

See 1 more Smart Citation

Human Skeleton Action Recognition Based on Monocular Depth Estimation

Wang

Zhang

Gu³

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Human action recognition exhibits limited accuracy in video surveillance due to the 2D information captured with the monocular cameras. To address the problem, a monocular depth estimation (SARDE)-based human skeleton action recognition method is proposed in this study. The proposed method comprises two tasks, i.e., human skeleton action recognition and monocular depth estimation, with the aim of transforming 2D human action into 3D. The two tasks are integrated with multi-task manner in an end-to-end training to fully utilize the correlation between action 1 recognition and depth estimation by sharing learning data to learn the depth features of the human skeleton joints more effectively for human action recognition. In this study, the graph-structured depth estimations methods with inception blocks and skip connections are investigated. The experimental results verify the effectiveness and superiority of the proposed model in skeleton action recognition, the model reaches state-of-the-art on the datasets.

show abstract

Section: Video Depth Estimationmentioning

confidence: 99%

“…The development of video surveillance is manifested by the automatic identification and analysis of the action of humans in the scene [1]. However, surveillance videos are primarily captured with 2D monocular cameras, such that action recognition tasks can only be performed from 2D information, thus causing low accuracy or efficiency.…”

Section: Introductionmentioning

confidence: 99%

Human Skeleton Action Recognition Based on Monocular Depth Estimation

Wang

Zhang

Gu³

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Appearance-based methods take face or eye images as input, and learn the mapping of face or eye images to gaze information by using a large number of training samples with ground-truth labels, thereby predicting the gaze information for new images using the trained model. Most methods are calibration-free, but some methods use a few calibration samples to optimize the model and reduce the impact of individual differences (Krafka et al, 2016 ; Gu et al, 2021 ; Liu G. et al, 2021 ; Wang et al, 2023 ). Liu G. et al ( 2021 ) trained a differential convolutional neural network to predict the gaze difference between two input eye images of a same subject, and the gaze direction of a new eye sample was predicted by inferring the gaze differences of a set of subject-specific calibration images.…”

Section: Introductionmentioning

confidence: 99%

A review on personal calibration issues for video-oculographic-based gaze tracking

Liu,

Chi,

Yang

2024

Front. Psychol.

View full text Add to dashboard Cite

Personal calibration is a process of obtaining personal gaze-related information by focusing on some calibration benchmarks when the user initially uses a gaze tracking system. It not only provides conditions for gaze estimation, but also improves gaze tracking performance. Existing eye-tracking products often require users to conduct explicit personal calibration first, thereby tracking and interacting based on their gaze. This calibration mode has certain limitations, and there is still a significant gap between theoretical personal calibration methods and their practicality. Therefore, this paper reviews the issues of personal calibration for video-oculographic-based gaze tracking. The personal calibration information in typical gaze tracking methods is first summarized, and then some main settings in existing personal calibration processes are analyzed. Several personal calibration modes are discussed and compared subsequently. The performance of typical personal calibration methods for 2D and 3D gaze tracking is quantitatively compared through simulation experiments, highlighting the characteristics of different personal calibration settings. On this basis, we discuss several key issues in designing personal calibration. To the best of our knowledge, this is the first review on personal calibration issues for video-oculographic-based gaze tracking. It aims to provide a comprehensive overview of the research status of personal calibration, explore its main directions for further studies, and provide guidance for seeking personal calibration modes that conform to natural human-computer interaction and promoting the widespread application of eye-movement interaction.

show abstract

“…Liu et al [ 25 ] propose a differential network (Diff-Nn) to address the gaze calibration problem by directly predicting the difference information between two images of the eyes. Gu et al [ 26 ] developed Diff-Nn for the gaze estimation using the left and right eye patch of one face simultaneously. Several other works mention that the performance based on the methods considering the difference information is directly affected by the number and the specific label of the inference image [ 25 , 26 ].…”

Section: Introductionmentioning

confidence: 99%

“…Gu et al [ 26 ] developed Diff-Nn for the gaze estimation using the left and right eye patch of one face simultaneously. Several other works mention that the performance based on the methods considering the difference information is directly affected by the number and the specific label of the inference image [ 25 , 26 ].…”

Section: Introductionmentioning

confidence: 99%

Gaze Estimation Approach Using Deep Differential Residual Network

Huang

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

Gaze estimation, which is a method to determine where a person is looking at given the person’s full face, is a valuable clue for understanding human intention. Similarly to other domains of computer vision, deep learning (DL) methods have gained recognition in the gaze estimation domain. However, there are still gaze calibration problems in the gaze estimation domain, thus preventing existing methods from further improving the performances. An effective solution is to directly predict the difference information of two human eyes, such as the differential network (Diff-Nn). However, this solution results in a loss of accuracy when using only one inference image. We propose a differential residual model (DRNet) combined with a new loss function to make use of the difference information of two eye images. We treat the difference information as auxiliary information. We assess the proposed model (DRNet) mainly using two public datasets (1) MpiiGaze and (2) Eyediap. Considering only the eye features, DRNet outperforms the state-of-the-art gaze estimation methods with angular-error of 4.57 and 6.14 using MpiiGaze and Eyediap datasets, respectively. Furthermore, the experimental results also demonstrate that DRNet is extremely robust to noise images.

show abstract

Gaze Estimation via a Differential Eyes’ Appearances Network with a Reference Grid

Cited by 15 publications

References 42 publications

Human Skeleton Action Recognition Based on Monocular Depth Estimation

Human Skeleton Action Recognition Based on Monocular Depth Estimation

A review on personal calibration issues for video-oculographic-based gaze tracking

Gaze Estimation Approach Using Deep Differential Residual Network

Contact Info

Product

Resources

About