LieToMe: An Ensemble Approach for Deception Detection from Facial Cues

Abstract

Deception detection is a relevant ability in high stakes situations such as police interrogatories or court trials, where the outcome is highly influenced by the interviewed person behavior. With the use of specific devices, e.g. polygraph or magnetic resonance, the subject is aware of being monitored and can change his behavior, thus compromising the interrogation result. For this reason, video analysis-based methods for automatic deception detection are receiving ever increasing interest. In this paper, a deception detection approach based on RGB videos, leveraging both facial features and stacked generalization ensemble, is proposed. First, a face, which is well-known to present several meaningful cues for deception detection, is identified, aligned, and masked to build video signatures. These signatures are constructed starting from five different descriptors, which allow the system to capture both static and dynamic facial characteristics. Then, video signatures are given as input to four base-level algorithms, which are subsequently fused applying the stacked generalization technique, resulting in a more robust meta-level classifier used to predict deception. By exploiting relevant cues via specific features, the proposed system achieves improved performances on a public dataset of famous court trials, with respect to other state-of-the-art methods based on facial features, highlighting the effectiveness of the proposed method.

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

“…Nowadays, intelligent computing and machine learning are the primary underlying technologies employed to develop automated algorithms and design learning strategies for finding the optimal solution to a large variety of tasks. [57][58][59][60][61] The state-of-the-art validates the incredible power of generative adversarial learning for image synthesis starting from text, sketch, or another image, as the information source. However, nowadays, the Wi-Fi signal is being explored for synthesizing visual data, opening up a new frontier for image synthesis and surveillance applications.…”

Lie Recognition with Multi-Modal Spatial–Temporal State Transition Patterns Based on Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory

“…The scenes from the top left corner to the right show the behavioral attitudes of lying people, while the scenes from the bottom left corner to the right show the behavioral attitudes of truth-telling people. Addressing the problem of learning human involuntary cues, recent research in the field of image processing/CV and machine learning reshapes computer engineering systems into machine learning-based (ML) systems [ 2 , 3 , 4 ]. ML-based systems can learn tiny facial marks [ 4 ] and behaviors in connection with body motion, as well as hand gestures [ 5 ], therefore making lie recognition suitable via CV and ML techniques.…”

“…Addressing the problem of learning human involuntary cues, recent research in the field of image processing/CV and machine learning reshapes computer engineering systems into machine learning-based (ML) systems [ 2 , 3 , 4 ]. ML-based systems can learn tiny facial marks [ 4 ] and behaviors in connection with body motion, as well as hand gestures [ 5 ], therefore making lie recognition suitable via CV and ML techniques. However, a combination of two or more human involuntary actions (known as cognitive cues) provides good results at some expenses [ 6 ].…”

“…The recognition accuracy of these methods [ 1 , 4 ] is low because some multi-modal involuntary cues and their complementary information are missing; thus, their accuracy needs to be improved since involuntary cues, such as those shown in Figure 1 , are significant factors in determining people’s behaviors while giving testimony during court trials or investigations. These involuntary cues are difficult to capture by using classical technique.…”

Low-Altitude Aerial Video Surveillance via One-Class SVM Anomaly Detection from Textural Features in UAV Images

“…In the last decades, vision-based systems are becoming increasingly important in supporting a wide range of application areas, including environment modelling for moving cameras [1][2][3][4][5], human action and event recognition [6][7][8][9], target and object detection [10][11][12][13][14][15][16]; even in areas such as medical image analysis [17][18][19][20][21][22][23], emotion or deception recognition [24][25][26][27][28][29], and immersive rehabilitation by serious games [30][31][32][33][34][35], these systems are, now, of daily use. At the same time, the last 10 years have seen substantial improvements of small-scale Unmanned Aerial Vehicles (UAVs), hereinafter UAVs, in terms of flight time, automatic control, embedded processing, and remote transmission.…”