Improving real-time CNN-based pupil detection through domain-specific data augmentation

Eivazi, Shaharam; Santini, Thiago; Keshavarzi, Alireza; Kübler, Thomas C.; Mazzei, Andrea

doi:10.1145/3314111.3319914

Cited by 32 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ideal solution would be to measure a full set of pupillometric features from the RGB cameras embedded on the robotic platform. Recent findings suggest that this approach could be feasible [43][44][45][46]74]; hence, we look forward to removing this limitation, making the system completely non-intrusive. Other than due to cognitive load changes, pupil dilation tends to be affected by other factors like excitement, stress, and environmental light conditions.…”

Section: Discussionmentioning

confidence: 99%

“…Both, mobile head mounted [39,40], and remote eye tracker [41,42] devices have been used as minimally invasive methods to measure pupillometric features, more appropriate for real-world scenarios. Recent research showed the possibility to measure TEPRs from RGB cameras, suitable for robotic platforms, making pupillometry a promising candidate to detect lies in real-life human-robot interactions [43][44][45][46].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Lies is a Child (Robot)’s Play: Gaze-Based Lie Detection in HRI

Pasquali

Gonzalez-Billandon

Aroyo

et al. 2021

Int J of Soc Robotics

View full text Add to dashboard Cite

Robots destined to tasks like teaching or caregiving have to build a long-lasting social rapport with their human partners. This requires, from the robot side, to be capable of assessing whether the partner is trustworthy. To this aim a robot should be able to assess whether someone is lying or not, while preserving the pleasantness of the social interaction. We present an approach to promptly detect lies based on the pupil dilation, as intrinsic marker of the lie-associated cognitive load that can be applied in an ecological human–robot interaction, autonomously led by a robot. We demonstrated the validity of the approach with an experiment, in which the iCub humanoid robot engages the human partner by playing the role of a magician in a card game and detects in real-time the partner deceptive behavior. On top of that, we show how the robot can leverage on the gained knowledge about the deceptive behavior of each human partner, to better detect subsequent lies of that individual. Also, we explore whether machine learning models could improve lie detection performances for both known individuals (within-participants) over multiple interaction with the same partner, and with novel partners (between-participant). The proposed setup, interaction and models enable iCub to understand when its partners are lying, which is a fundamental skill for evaluating their trustworthiness and hence improving social human–robot interaction.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Detecting Lies is a Child (Robot)’s Play: Gaze-Based Lie Detection in HRI

Pasquali

Gonzalez-Billandon

Aroyo

et al. 2021

Int J of Soc Robotics

View full text Add to dashboard Cite

show abstract

“…Since then, several approaches like (Fuhl et al, 2015(Fuhl et al, , 2016Santini et al, 2018aSantini et al, , 2018b were proposed for robust real-time pupil detection in challenging natural environments like driving and walking. Current state-ofthe-art approach (Eivazi et al, 2019) reported a pupil detection rate of ~85% on PupilNet (Fuhl et al, 2017) and LPW (Tonsen et al, 2016) datasets and a detection rate of ~74% on Swirski (Świrski et al, 2012) dataset. Yet, the performance of these approaches in terms of gaze estimation in similar challenging environments with such pupil detection accuracies is still unknown.…”

Section: Related Workmentioning

confidence: 99%

I2DNet - Design and real-time evaluation of an appearance-based gaze estimation system

Murthy

Brahmbhatt

Arjun

et al. 2021

JEMR

View full text Add to dashboard Cite

Gaze estimation problem can be addressed using either model-based or appearance-based approaches. Model-based approaches rely on features extracted from eye images to fit a 3D eye-ball model to obtain gaze point estimate while appearance-based methods attempt to directly map captured eye images to gaze point without any handcrafted features. Recently, availability of large datasets and novel deep learning techniques made appearance-based methods achieve superior accuracy than model-based approaches. However, many appearance-based gaze estimation systems perform well in within-dataset validation but fail to provide the same degree of accuracy in cross-dataset evaluation. Hence, it is still unclear how well the current state-of-the-art approaches perform in real-time in an interactive setting on unseen users. This paper proposes I2DNet, a novel architecture aimed to improve subject-independent gaze estimation accuracy that achieved a state-of-the-art 4.3 and 8.4 degree mean angle error on the MPIIGaze and RT-Gene datasets respectively. We have evaluated the proposed system as a gaze-controlled interface in real-time for a 9-block pointing and selection task and compared it with Webgazer.js and OpenFace 2.0. We have conducted a user study with 16 participants, and our proposed system reduces selection time and the number of missed selections statistically significantly compared to other two systems.

show abstract

“…Eivazi et al [13] augmented a dataset of real eye images by recording scene re ections o the anterior side of black coated glasses and superimposing them on the real eye imagery. To achieve similar re ections, we incorporate a 3 mm thick eyeglasses with black frames.…”

Section: Rendered Datasetsmentioning

confidence: 99%

“…Both models are trained using the loss function strategy proposed in RITnet [9]. is strategy involves using a weighted combination of four loss functions: Image Augmentation: Image augmentation aids in broadening the statistical distribution of information content and combats overing [13]. Previous e orts [9,27] have shown that data augmentation on eye images improves the performance of convolutional networks under naturalistic conditions such as varying contrast, eye makeup, eyeglasses, multiple re ections and image distortions.…”

Section: Model Architecturementioning

confidence: 99%

RIT-Eyes: Rendering of near-eye images for eye-tracking applications

Nair,

Kothari,

Chaudhary

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Improving real-time CNN-based pupil detection through domain-specific data augmentation

Cited by 32 publications

References 21 publications

Detecting Lies is a Child (Robot)’s Play: Gaze-Based Lie Detection in HRI

Detecting Lies is a Child (Robot)’s Play: Gaze-Based Lie Detection in HRI

I2DNet - Design and real-time evaluation of an appearance-based gaze estimation system

RIT-Eyes: Rendering of near-eye images for eye-tracking applications

Contact Info

Product

Resources

About