A3D: A Device for Studying Gaze in 3D

Qodseya, Mahmoud; Sanzari, Marta; Ntouskos, Valsamis; Pirri, Fiora

doi:10.1007/978-3-319-46604-0_41

Cited by 7 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It inquires the motion capture tabletop objects' poses in a moment and computes the intersections between the object models and gaze vector; however, the settings are costly and the model does not scale to larger environments. For wearable 3D gaze acquisition outside the laboratory, Qodseya et al [23] and Hausamann et al [24] developed eye-trackers equipped with depth sensors. They overlay 2D gaze points on the depth image and directly reconstruct the 3D human gaze.…”

Section: Dynamic Object Handling For 4d Attention Mappingmentioning

confidence: 99%

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

Oishi

Koide

Yokozuka

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

This study presents a framework for capturing human attention in the spatio-temporal domain using eyetracking glasses. Attention mapping is a key technology for human perceptual activity analysis or Human-Robot Interaction (HRI) to support human visual cognition; however, measuring human attention in dynamic environments is challenging owing to the difficulty in localizing the subject and dealing with moving objects. To address this, we present a comprehensive framework, 4D Attention, for unified gaze mapping onto static and dynamic objects. Specifically, we estimate the glasses pose by leveraging a loose coupling of direct visual localization and Inertial Measurement Unit (IMU) values. Further, by installing reconstruction components into our framework, dynamic objects not captured in the 3D environment map are instantiated based on the input images. Finally, a scene rendering component synthesizes a first-person view with identification (ID) textures and performs direct 2D-3D gaze association. Quantitative evaluations showed the effectiveness of our framework. Additionally, we demonstrated the applications of 4D Attention through experiments in real situations 1 . 4D attention Camera inputRendered scene ID Object mask

show abstract

Section: Dynamic Object Handling For 4d Attention Mappingmentioning

confidence: 99%

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

Oishi

Koide

Yokozuka

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…There has been many gaze estimation studies where the camera has access to the eyes or face of a person whose gaze is predicted [25,31,35,46,49]. Model-based methods often use a geometric prior information, e.g., pupil center [19] or iris contours [36] to create an eye model to predict the gaze.…”

Section: Related Workmentioning

confidence: 99%

“…There is many evidence in the literature showing that eye gaze [10,25,34] (as well as its approximation in terms of head pose [7,8,11,26,33,53]) is an important cue, e.g., for the detection of social interactions. However, robust estimation of the eye gaze in unconstrained scenarios is only possible by using relatively expensive equipment (e.g., eye tracker [15,[42][43][44]) or by equipping a constrained area with cameras and other sensors, thus limiting the activity extent of the subject [25,31,35,46,49].…”

Section: Introductionmentioning

confidence: 99%

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Thakur

Beyan

Morerio

et al. 2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Gaze prediction in egocentric videos is a fairly new research topic, which might have several applications for assistive technology (e.g., supporting blind people in their daily interactions), security (e.g., attention tracking in risky work environments), education (e.g., augmented / mixed reality training simulators, immersive games) and so forth. Egocentric gaze is typically estimated from video while few works attempt to use inertial measurement unit (IMU) data, a sensor modality often available in wearable devices (e.g., augmented reality headsets). Instead, in this paper, we examine whether joint learning of egocentric video and corresponding IMU data can improve the first-person gaze prediction compared to using these modalities separately. In this respect, we propose a multimodal network and evaluate it on several unconstrained social interaction scenarios captured by a first-person perspective. The proposed multimodal network achieves better results compared to unimodal methods as well as several (multimodal) baselines, showing that using egocentric video together with IMU data can boost the first-person gaze estimation performance.

show abstract

“…During calibration, the subject needs to stare at the specific reference marker and sometimes, such active personal calibration interrupts user-scene interactions. Although the calibration procedure has become much simpler and the number of calibration markers has been reduced down to one in some works [12]; it still requires the user to participate in the calibration task actively.…”

Section: Introductionmentioning

confidence: 99%

3D Gaze Estimation for Head-Mounted Eye Tracking System With Auto-Calibration Method

Liu

2020

IEEE Access

View full text Add to dashboard Cite

The general challenges of 3D gaze estimation for head-mounted eye tracking systems are inflexible marker-based calibration procedure and significant errors of depth estimation. In this paper, we propose a 3D gaze estimation with an auto-calibration method. To acquire the accurate 3D structure of the environment, an RGBD camera is applied as the scene camera of our system. By adopting the saliency detection method, saliency maps can be acquired through scene images, and 3D salient pixels in the scene are considered potential 3D calibration targets. The 3D eye model is built on the basis of eye images to determine gaze vectors. By combining 3D salient pixels and gaze vectors, the auto-calibration can be achieved with our calibration method. Finally, the 3D gaze point is obtained through the calibrated gaze vectors, and the point cloud is generated from the RGBD camera. The experimental result shows that the proposed system can achieve an average accuracy of 3.7 • in the range of 1 m to 4 m indoors and 4.0 • outdoors. The proposed system also presents a great improvement in depth measurement, which is sufficient for tracking users' visual attention in real scenes. INDEX TERMS Head-mounted gaze tracking system, saliency maps, auto-calibration, 3D gaze estimation.

show abstract

A3D: A Device for Studying Gaze in 3D

Cited by 7 publications

References 39 publications

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

4D Attention: Comprehensive Framework for Spatio-Temporal Gaze Mapping

Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

3D Gaze Estimation for Head-Mounted Eye Tracking System With Auto-Calibration Method

Contact Info

Product

Resources

About