MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality

Abstract

With the emergence of Virtual and Mixed Reality (XR) devices, eye tracking has received significant attention in the computer vision community. Eye gaze estimation is a crucial component in XRenabling energy efficient rendering, multi-focal displays, and effective interaction with content. In head-mounted XR devices, the eyes are imaged off-axis to avoid blocking the field of view. This leads to increased challenges in inferring eye related quantities and simultaneously provides an opportunity to develop accurate and robust learning based approaches. To this end, we present MagicEyes, the first large scale eye dataset collected using real MR devices with comprehensive ground truth labeling. MagicEyes includes 587 subjects with 80, 000 images of human-labeled ground truth and over 800, 000 images with gaze target labels. We evaluate several state-of-the-art methods on MagicEyes and also propose a new multi-task EyeNet model designed for detecting the cornea, glints and pupil along with eye segmentation in a single forward pass.

EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes

“…While there are several publicly available eye image datasets, they are unfortunately not directly suited for our purposes of view, gaze, and illumination synthesis. The majority of these datasets are tailored for the task of gaze-tracking [Fuhl et al 2021;Fusek 2018;Kim et al 2019;Tonsen et al 2017;Wood et al 2015;Wu et al 2020;] while others cater to different problems such as pupil detection [Tonsen et al 2016], eye closure detection [Song et al 2014] or eyelash segmentation [Xiao et al 2021]. While there are datasets that aim at modeling high-quality eyes and periocular region [Bérard et al 2019[Bérard et al , 2014, these are not suited for relighting purposes.…”

Improving the Deeplabv3+ Model with Attention Mechanisms Applied to Eye Detection and Segmentation

“…The proposed dataset is established based on existing datasets including Asia [17] and Ubiris [18,19] (for iris recognition), POG [16] and NVGaze [14] (for focusing of the eyes), and GIW [12] and BAY [23] (for eye movement types). In previous studies, GAN [24], 550 K [25], LPW [21], and Else [20] have also been used as they are related to the pupil and iris of the eye.…”

“…In eye detection, the eye image database is essential. For example, datasets are used for eye focus estimation [11][12][13], eye tracking [14][15][16], studying the iris of the eye [17][18][19], and investigating the pupil [20][21][22].…”

TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types

“…--1 25Hz 384 × 288 866,069 Y -Y ----------------MEMD [41] --1 25Hz 384 × 288 866,069 ---------------Y Y Y -ME [96] 587 1 1 --640 × 480 880,000 Y Y Y -----------Y ----OpenEDS [63] 152 1 --200Hz 400 × 640 356,649 Y Y Y ----------------GIW [72] --1 120Hz 640 × 480 ≈2,016,000…”