Point of gaze estimation using corneal surface reflection and omnidirectional camera image

Ogawa, Taishi; Nakazawa, Atsushi; Nishida, Toyoaki

doi:10.23919/mva.2017.7986842

Cited by 3 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, their proposed model could be able to determine the location and circumstances of the person at the time of image capturing. Environment map obtained from the corneal reflection enables several applications such as scene panorama reconstruction (Nakazawa, Nitschke, & Nishida, 2016), biometrics (Bowyer, Hollingsworth & Flynn, 2008), creation of a 3D model (Nishino & Nayar, 2006), super-resolution imaging , illumination normalization in face recognition , point of gaze estimation (Kar & Corcoran, 2017;Nitschke, Nakazawa & Nishida, 2013;Ogawa, Nakazawa & Nishida, 2018), etc. A study was conducted by Wan and his co-authors to remove reflection components from a mixture image to recover reflection scenes from an image (Wan, Shi, Li, Duan & Kot, 2020).…”

Section: Literature Reviewmentioning

confidence: 99%

Crime Detection and Criminal Recognition to Intervene in Interpersonal Violence Using Deep Convolutional Neural Network With Transfer Learning

Haque

Hafiz

Azad

et al. 2021

International Journal of Ambient Computing and Intelligence

View full text Add to dashboard Cite

Interpersonal violence, such as physical and sexual abuse, eve-teasing, bullying, and taking hostages, is a growing concern in our society. The criminals who directly or indirectly committed the crime often do not go into the trial for the lack of proper evidence as it is very tough to collect photographic proof of the incident. A subject's corneal reflection has the potentiality to reveal the bystander images. Motivated with this clue, a novel approach is proposed in the current paper that uses a convolutional neural network (CNN) along with transfer learning in identifying crime as well as recognizing the criminals from the corneal reflected image of the victim called the Purkinje image. This study found that off-the-shelf CNN can be fine-tuned to extract discriminative features from very low resolution and noisy images. The procedure is validated using the developed datasets comprising six different subjects taken at diverse situations. They confirmed that it has the ability to recognize criminals from corneal reflection images with an accuracy of 95.41%.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Crime Detection and Criminal Recognition to Intervene in Interpersonal Violence Using Deep Convolutional Neural Network With Transfer Learning

Haque

Hafiz

Azad

et al. 2021

International Journal of Ambient Computing and Intelligence

View full text Add to dashboard Cite

show abstract

“…In these works, a camera capturing an image of the spherical or aspherical eye that exhibits corneal reflections is modeled as a non-rigid catadioptric imaging system [15]. Applying this model enables the scene illumination to be reconstructed from an eye image, such as through geometric calibration between the eye and a computer display [16], optical see-through head-mounted displays (OST-HMD) [17], or a fish-eye camera [18]. In [19], an aspherical surface model was introduced for the cornea.…”

Section: A Eye Image Analysismentioning

confidence: 99%

Visual Place Recognition From Eye Reflection

et al. 2021

Self Cite

View full text Add to dashboard Cite

The cornea in the human eye reflects incoming environmental light, which means we can obtain information about the surrounding environment from the corneal reflection in facial images. In recent years, as the quality of consumer cameras increases, this has caused privacy concerns in terms of identifying the people around the subject or where the photo is taken. This paper investigates the security risk of eye corneal reflection images: specifically, visual place recognition from eye reflection images. First, we constructed two datasets containing pairs of scene and corneal reflection images. The first dataset is taken in a virtual environment. We showed pre-captured scene images in a 180-degree surrounding display system and took corneal reflections from subjects. The second dataset is taken in an outdoor environment. We developed several visual place recognition algorithms, including CNN-based image descriptors featuring a naive Siamese network and AFD-Net combined with entire image feature representations including VLAD and NetVLAD, and compared the results. We found that AFD-Net+VLAD performed the best and was able to accurately determine the scene in 73.08% of the top-five candidate scenes. These results demonstrate the potential to estimate the location at which a facial picture was taken, which simultaneously leads to a) positive applications such as the localization of a robot while conversing with persons and b) negative scenarios including the security risk of uploading facial images to the public.

show abstract

Point of Gaze Estimation Using Corneal Surface Reflection and Omnidirectional Camera Image

Ogawa

Nakazawa

Nishida

2018

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

Taishi OGAWA †a) , Atsushi NAKAZAWA †b) , Nonmembers, and Toyoaki NISHIDA †, † †c) , Member SUMMARYWe present a human point of gaze estimation system using corneal surface reflection and omnidirectional image taken by spherical panorama cameras, which becomes popular recent years. Our system enables to find where a user is looking at only from an eye image in a 360 • surrounding scene image, thus, does not need gaze mapping from partial scene images to a whole scene image that are necessary in conventional eye gaze tracking system. We first generate multiple perspective scene images from an omnidirectional (equirectangular) image and perform registration between the corneal reflection and perspective images using a corneal reflection-scene image registration technique. We then compute the point of gaze using a corneal imaging technique leveraged by a 3D eye model, and project the point to an omnidirectional image. The 3D eye pose is estimate by using the particle-filter-based tracking algorithm. In experiments, we evaluated the accuracy of the 3D eye pose estimation, robustness of registration and accuracy of PoG estimations using two indoor and five outdoor scenes, and found that gaze mapping error was 5.546 [deg] on average.

show abstract

Point of gaze estimation using corneal surface reflection and omnidirectional camera image

Cited by 3 publications

References 6 publications

Crime Detection and Criminal Recognition to Intervene in Interpersonal Violence Using Deep Convolutional Neural Network With Transfer Learning

Crime Detection and Criminal Recognition to Intervene in Interpersonal Violence Using Deep Convolutional Neural Network With Transfer Learning

Visual Place Recognition From Eye Reflection

Point of Gaze Estimation Using Corneal Surface Reflection and Omnidirectional Camera Image

Contact Info

Product

Resources

About