SLAM-based localization of 3D gaze using a mobile eye tracker

Wang, Haofei; Pi, Jimin; Qin, Tong; Shen, Shaojie; Shi, Bertram E.

doi:10.1145/3204493.3204584

Cited by 36 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It contains rich information of human intent that enables researchers to gain insights into human cognition [1], [2] and behavior [3], [4]. It is widely demanded by various applications, e.g., human-computer interaction [5], [6], [7] and head-mounted devices [8], [9], [10]. To enable such applications, accurate gaze estimation methods are critical.…”

Section: Introductionmentioning

confidence: 99%

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Gaze estimation reveals where a person is looking. It is an important clue for understanding human intention. The recent development of deep learning has revolutionized many computer vision tasks, the appearance-based gaze estimation is no exception. However, it lacks a guideline for designing deep learning algorithms for gaze estimation tasks. In this paper, we present a comprehensive review of the appearance-based gaze estimation methods with deep learning. We summarize the processing pipeline and discuss these methods from four perspectives: deep feature extraction, deep neural network architecture design, personal calibration as well as device and platform. Since the data pre-processing and post-processing methods are crucial for gaze estimation, we also survey face/eye detection method, data rectification method, 2D/3D gaze conversion method, and gaze origin conversion method. To fairly compare the performance of various gaze estimation approaches, we characterize all the publicly available gaze estimation datasets and collect the code of typical gaze estimation algorithms. We implement these codes and set up a benchmark of converting the results of different methods into the same evaluation metrics. This paper not only serves as a reference to develop deep learning-based gaze estimation methods but also a guideline for future gaze estimation research. Implemented methods and data processing codes are available at http://phi-ai.org/GazeHub.

show abstract

Section: Introductionmentioning

confidence: 99%

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Cheng,

Wang,

Bao

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, these types of eye trackers are expensive and generally used in a laboratory setting together with a chin rest or bite bar that stabilizes the head. To reduce the restriction on head movement, the authors proposed different solutions [19,20]. However, these eye trackers do not work well in outdoor situations because of the sun's infrared radiation.…”

Section: Related Workmentioning

confidence: 99%

Gaze Tracking Using an Unmodified Web Camera and Convolutional Neural Network

2021

View full text Add to dashboard Cite

Gaze estimation plays a significant role in understating human behavior and in human–computer interaction. Currently, there are many methods accessible for gaze estimation. However, most approaches need additional hardware for data acquisition which adds an extra cost to gaze tracking. The classic gaze tracking approaches usually require systematic prior knowledge or expertise for practical operations. Moreover, they are fundamentally based on the characteristics of the eye region, utilizing infrared light and iris glint to track the gaze point. It requires high-quality images with particular environmental conditions and another light source. Recent studies on appearance-based gaze estimation have demonstrated the capability of neural networks, especially convolutional neural networks (CNN), to decode gaze information present in eye images and achieved significantly simplified gaze estimation. In this paper, a gaze estimation method that utilizes a CNN for gaze estimation that can be applied to various platforms without additional hardware is presented. An easy and fast data collection method is used for collecting face and eyes images from an unmodified desktop camera. The proposed method registered good results; it proves that it is possible to predict the gaze with reasonable accuracy without any additional tools.

show abstract

“…Tessid et al [27] estimate spatial and temporal visual attention using a head-worn IMU when the performed actions are grabbing objects, opening doors, writing on a whiteboard, etc. Wang et al [45] propose an approach to predict 3D-gaze locations using head pose estimates from a SLAM kit along a RGB-D sensor to perform gaze estimation in human-robot interfaces. Their method [45] was tested on very constraint scenarios in which subjects gaze at a fixation cross.…”

Section: Gaze Prediction With Imu Datamentioning

confidence: 99%

“…Wang et al [45] propose an approach to predict 3D-gaze locations using head pose estimates from a SLAM kit along a RGB-D sensor to perform gaze estimation in human-robot interfaces. Their method [45] was tested on very constraint scenarios in which subjects gaze at a fixation cross.…”

Section: Gaze Prediction With Imu Datamentioning

confidence: 99%

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Thakur

Beyan

Morerio

et al. 2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Gaze prediction in egocentric videos is a fairly new research topic, which might have several applications for assistive technology (e.g., supporting blind people in their daily interactions), security (e.g., attention tracking in risky work environments), education (e.g., augmented / mixed reality training simulators, immersive games) and so forth. Egocentric gaze is typically estimated from video while few works attempt to use inertial measurement unit (IMU) data, a sensor modality often available in wearable devices (e.g., augmented reality headsets). Instead, in this paper, we examine whether joint learning of egocentric video and corresponding IMU data can improve the first-person gaze prediction compared to using these modalities separately. In this respect, we propose a multimodal network and evaluate it on several unconstrained social interaction scenarios captured by a first-person perspective. The proposed multimodal network achieves better results compared to unimodal methods as well as several (multimodal) baselines, showing that using egocentric video together with IMU data can boost the first-person gaze estimation performance.

show abstract

SLAM-based localization of 3D gaze using a mobile eye tracker

Cited by 36 publications

References 20 publications

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark

Gaze Tracking Using an Unmodified Web Camera and Convolutional Neural Network

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Contact Info

Product

Resources

About