I2DNet - Design and real-time evaluation of an appearance-based gaze estimation system

Abstract

Gaze estimation problem can be addressed using either model-based or appearance-based approaches. Model-based approaches rely on features extracted from eye images to fit a 3D eye-ball model to obtain gaze point estimate while appearance-based methods attempt to directly map captured eye images to gaze point without any handcrafted features. Recently, availability of large datasets and novel deep learning techniques made appearance-based methods achieve superior accuracy than model-based approaches. However, many appearance-based gaze estimation systems perform well in within-dataset validation but fail to provide the same degree of accuracy in cross-dataset evaluation. Hence, it is still unclear how well the current state-of-the-art approaches perform in real-time in an interactive setting on unseen users. This paper proposes I2DNet, a novel architecture aimed to improve subject-independent gaze estimation accuracy that achieved a state-of-the-art 4.3 and 8.4 degree mean angle error on the MPIIGaze and RT-Gene datasets respectively. We have evaluated the proposed system as a gaze-controlled interface in real-time for a 9-block pointing and selection task and compared it with Webgazer.js and OpenFace 2.0. We have conducted a user study with 16 participants, and our proposed system reduces selection time and the number of missed selections statistically significantly compared to other two systems.

PerimetryNet: A multiscale fine grained deep network for three‐dimensional eye gaze estimation using visual field analysis

“…11 Dilated convolution mechanism has also been applied with eye images and face image as input, where dilated convolutions are used to instead of several max pooling layers in their model. 13,14 An extended dilated convolutional approach has introduced a gaze decomposition method that decomposes the gaze angle into the sum of a subject-independent gaze estimate from the image and a subject-dependent bias. 15 AGE-Net has tried to use an attention mechanism with the dilated convolution model.…”

“…We compare our work with prior works 11,12,[14][15][16][35][36][37][43][44][45][46] that use face images or face plus eye images as input on the MPIIGaze and EYEDIAP datasets. The results are shown in Table 2.…”

“…The reference point for image normalization is set to the center of the face. We follow same procedure as state-of-the-art works 11,12,[14][15][16]35,36,43,44 used for cross-subject validation.…”

“…9 In addition, the application of deep convolutional neural networks (CNNs) has reduced estimation error of appearance-based system significantly. 10 In order to conduct high-quality gaze estimation, previous works have tried to use multistream input, [11][12][13][14][15] dilated convolution, [13][14][15] multitask techniques, 16,17 unsupervised representation learning, 18 and zero-shot learning. 19 Gaze information is directly conducted as a regression result from estimation.…”

“…In order to conduct high‐quality gaze estimation, previous works have tried to use multistream input, 11‐15 dilated convolution, 13‐15 multitask techniques, 16,17 unsupervised representation learning, 18 and zero‐shot learning 19 . Gaze information is directly conducted as a regression result from estimation.…”