Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images

Abstract

Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance.

P‐4.27: Modeling Eye Movement and Reflection in Virtual Environments for Eye Tracking

“…Each of these methods offers unique benefits and challenges, influencing their application in VR environments. Recent advancements in gaze estimation, particularly the use of appearance-based methods [4], have shown significant potential in VR settings. These methods, leveraging synthesized images of the eye region, enable more realistic simulations of eye movements [4].…”

“…Recent advancements in gaze estimation, particularly the use of appearance-based methods [4], have shown significant potential in VR settings. These methods, leveraging synthesized images of the eye region, enable more realistic simulations of eye movements [4]. Furthermore, the study by Wood et al [5] provides a comprehensive overview of appearance-based gaze estimation using deep learning, offering insights into the complexities and challenges associated with simulating realistic eye movements in digital environments [5].…”

Computer Vision and Machine Learning for Intelligent Sensing Systems

“…Oh et al [10] suggested estimating gaze by detecting eye region landmarks through a single eye image. It learns representations of images at various resolutions, and the self-attention module is used to obtain a refined feature map with better spatial information.…”