In the Eye of the Deceiver: Analyzing Eye Movements as a Cue to Deception

Abstract

Deceit occurs in daily life and, even from an early age, children can successfully deceive their parents. Therefore, numerous book and psychological studies have been published to help people decipher the facial cues to deceit. In this study, we tackle the problem of deceit detection by analyzing eye movements: blinks, saccades and gaze direction. Recent psychological studies have shown that the non-visual saccadic eye movement rate is higher when people lie. We propose a fast and accurate framework for eye tracking and eye movement recognition and analysis. The proposed system tracks the position of the iris, as well as the eye corners (the outer shape of the eye). Next, in an offline analysis stage, the trajectory of these eye features is analyzed in order to recognize and measure various cues which can be used as an indicator of deception: the blink rate, the gaze direction and the saccadic eye movement rate. On the task of iris center localization, the method achieves within pupil localization in 91.47% of the cases. For blink localization, we obtained an accuracy of 99.3% on the difficult EyeBlink8 dataset. In addition, we proposed a novel metric, the normalized blink rate deviation to stop deceitful behavior based on blink rate. Using this metric and a simple decision stump, the deceitful answers from the Silesian Face database were recognized with an accuracy of 96.15%.

Pupil Localisation and Eye Centre Estimation Using Machine Learning and Computer Vision

“…To maintain the reliability in our performance measure, we utilised different metrics as well as the newly introduced S ED in this work. Table 2 summarises the results achieved from the proposed approach using wec and bec metrics, which have been used in recent similar works [35][36][37][42][43][44]. We are specifically interested in the wec measure when error ≤0.05, which indicates the model estimation within the pupil diameter (i.e., more restricted).…”

“…Furthermore, the validation is performed on a dataset containing artificially rendered images, which in most cases do not reflect the real-time dynamics. Likewise, [36] presented gaze estimation that utilises the DL-based facial landmarks detection following the image segmentation to identify the pupil within the input images. However, the 81% accuracy produced by the algorithm on a benchmark dataset indicates the lack of preciseness in pupil localisation that might lead to the incorrect gaze estimation.…”

“…To validate the proposed methodology and reliable performance measure, we used three different publicly available datasets. The first dataset is known as Talking-Face [41] and has been used in previous works [36]. This dataset contains 5000 video frames captured during an engaged conversation with a person for 200 s. The original objective of this dataset was to model the facial behaviour during a natural conversation.…”

“…In contrast to Talking-Face, we used the BIO-ID dataset [42], which is comparatively more challenging and has been used as a benchmark in various relevant studies such as [36,40]. The data were acquired from 23 different subjects during multiple sessions and have 1521 images in total containing varying gaze positions, illuminations, background scenes, eye features (e.g., eye colour, gender, ethnicity, iris size), camera focus, and hence eye-frame (and face rectangle) size.…”

Weakly-Supervised Degree of Eye-Closeness Estimation

“…Several eye-based systems have been proposed in the literature which use the percent of closeness (PERCLOSE) and average eye closure speed (AECS) measures for different decisions, such as drowsiness detection where PER-CLOSE increases [8,10,28,30,21,9,26,7,5] and AECS decreases [12,3,4], for a drowsy driver. Existing eye-based approaches mostly use eye and face detectors, such as Viola Jones algorithm [33], and detect the eye state using classical computer vision techniques.…”