Gaze Estimation Approach Using Deep Differential Residual Network

Huang, Longzhao; Li, Yujie; Wang, Xu; Wang, Haoyu; Bouridane, Ahmed; Chaddad, Ahmad

doi:10.3390/s22145462

Cited by 8 publications

(4 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multimodal CNN [18] y n Low complexity Gazemap [20] y n Robustness to head pose and image quality Multiview CNN [21] y n Multitask solution Differential NN [22] y n Less calibration DRNet [23] y n Robustness to noise U-Train [24] y n Unsupervised Spatial weights CNN [16] n y Robustness to facial appearance variation BPA-Net [25] n y Robustness to facial appearance variation Recurrent CNN [17] n y Temporal modality DEA-Net [26] y n Less samples iTracker [27] y y High generalization in different datasets Bi-LSTM [28] y y Low complexity and robustness to resolution Gaze360 [29] n y High generalization in real scene GEDD-Net [30] y y low complexity high performance calibration STTDN [31] y y feature fusion and dynamic feature extraction FreeGaze (Ours) y y Improved normalization method and landmarks' impact on gaze estimation…”

Section: Eye Face Advantagesmentioning

confidence: 99%

“…Multimodal CNN [18] 6.3 -Spatial weights CNN [16] 4.8 6.0 Dilated-Convolutions [35] 4.8 -Recurrent CNN [17] -3.4 L2CS-Net [36] 3.92 -Bi-LSTM [28] 4.18 5.84 CA-Net [37] 4.1 5.3 FARE-Net [38] 4.3 5.71 DEA-Net [26] 4.38 -GEDD-Net [30] 4.5 5.4 STTDN [31] 3.73 5.02 U-Train [24] -6.79 DRNet [23] 4.57 6.14 FreeGaze 3.11 2.75…”

Section: Mpiigaze Eyediapmentioning

confidence: 99%

“…Liu et al proposed directly training a differential network to predict the gaze difference between two eyes' gazes of one subject [22]. Huang et al proposed a differential residual model (DRNet) combined with a new loss function for gaze estimation using the difference information of two eye images [23]. Yu et al proposed an unsupervised representation learning for gaze estimation using eye images, addressing the issue of difficulty in collecting large and diverse data [24].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

FreeGaze: A Framework for 3D Gaze Estimation Using Appearance Cues from a Facial Video

Tian,

Tu,

et al. 2023

Sensors

View full text Add to dashboard Cite

Gaze is a significant behavioral characteristic that can be used to reflect a person’s attention. In recent years, there has been a growing interest in estimating gaze from facial videos. However, gaze estimation remains a challenging problem due to variations in appearance and head poses. To address this, a framework for 3D gaze estimation using appearance cues is developed in this study. The framework begins with an end-to-end approach to detect facial landmarks. Subsequently, we employ a normalization method and improve the normalization method using orthogonal matrices and conduct comparative experiments to prove that the improved normalization method has a higher accuracy and a lower computational time in gaze estimation. Finally, we introduce a dual-branch convolutional neural network, named FG-Net, which processes the normalized images and extracts eye and face features through two branches. The extracted multi-features are then integrated and input into a fully connected layer to estimate the 3D gaze vectors. To evaluate the performance of our approach, we conduct ten-fold cross-validation experiments on two public datasets, namely MPIIGaze and EyeDiap, achieving remarkable accuracies of 3.11° and 2.75°, respectively. The results demonstrate the high effectiveness of our proposed framework, showcasing its state-of-the-art performance in 3D gaze estimation.

show abstract

Section: Eye Face Advantagesmentioning

confidence: 99%

Section: Mpiigaze Eyediapmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FreeGaze: A Framework for 3D Gaze Estimation Using Appearance Cues from a Facial Video

Tian,

Tu,

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Another notable gaze estimation model, SAGE (fast accurate gaze tracker) [10], combines eye images and landmarks, demonstrating high effectiveness. These and many other types of research have been done on gaze estimation from facial images [11][12][13]. However, a significant obstacle to furthering these models is the limited availability of resources.…”

Section: Introductionmentioning

confidence: 99%

Advancements in Gaze Coordinate Prediction Using Deep Learning: A Novel Ensemble Loss Approach

Kim,

Lee,

Lee

2024

Applied Sciences

View full text Add to dashboard Cite

Recent advancements in deep learning have enabled gaze estimation from images of the face and eye areas without the need for precise geometric locations of the eyes and face. This approach eliminates the need for complex user-dependent calibration and the issues associated with extracting and tracking geometric positions, making further exploration of gaze position performance enhancements challenging. Motivated by this, our study focuses on an ensemble loss function that can enhance the performance of existing 2D-based deep learning models for gaze coordinate (x, y) prediction. We propose a new function and demonstrate its effectiveness by applying it to models from prior studies. The results show significant performance improvements across all cases. When applied to ResNet and iTracker models, the average absolute error reduced significantly from 7.5 cm to 1.2 cm and from 7.67 cm to 1.3 cm, respectively. Notably, when implemented on the AFF-Net, which boasts state-of-the-art performance, the average absolute error was reduced from 4.21 cm to 0.81 cm, based on our MPIIFaceGaze dataset. Additionally, predictions for ranges never encountered during the training phase also displayed a very low error of 0.77 cm in terms of MAE without any personalization process. These findings suggest significant potential for accuracy improvements while maintaining computational complexity similar to the existing models without the need for creating additional or more complex models.

show abstract

Gaze estimation using convolutional neural networks

Karmi,

Rahmany,

Khlifa

2023

SIViP

View full text Add to dashboard Cite

Numerous investigations on gaze estimate techniques for analyzing human behavior have been made in recent years. The majority of which have focused on gaze tracking techniques. This article proposes a new method for gaze estimation. The proposed system is divided into three phases: (i) Estimation of head position using Convolutional neural networks CNN (VGG16, Resnet50, InceptionV3), (ii) Detection of eyes area using Viola Jones' algorithm, and in phase (iii) gaze estimation using three different models: pre-trained CNN, CNN from scratch, as well as Bilinear Convolutional Neural Networks (B-CNNs). Columbia gaze database is used in the validation experiments. When compared to earlier efforts, the experimental results demonstrate that the proposed method produces a more exact outcome.

show abstract

Gaze Estimation Approach Using Deep Differential Residual Network

Cited by 8 publications

References 39 publications

FreeGaze: A Framework for 3D Gaze Estimation Using Appearance Cues from a Facial Video

FreeGaze: A Framework for 3D Gaze Estimation Using Appearance Cues from a Facial Video

Advancements in Gaze Coordinate Prediction Using Deep Learning: A Novel Ensemble Loss Approach

Gaze estimation using convolutional neural networks

Contact Info

Product

Resources

About