Automated Deception Detection of Males and Females From Non-Verbal Facial Micro-Gestures

Abstract

Gender bias within Artificial intelligence driven systems is currently a hot topic and is one of a number of areas where the data used to train, validate and test machine learning algorithms is under more scrutiny than ever before. In this paper we investigate if there is a difference between the nonverbal cues to deception generated by males and females through the use of an automated deception detection system. The system uses hierarchical neural networks to extract 36 channels of non-verbal head and facial behaviors whilst male and female participants are engaged in either a deceptive or truthful roleplaying task. An Image Vector dataset, comprising of 86584 vectors, is collated which uses a fixed sliding window slot of 1 second to record deceptive or truthful slots. Experiments were conducted on three variants of the dataset, all males, all females and mixed in order to examine if the differences in cues generated by males and females lead to differences in the accuracies of machine learning algorithms which classify their behavior. Results showed differences in nonverbal cues between males and females, with both genders at a disadvantage when treated by classifiers trained on both genders rather than classifiers specifically trained for each gender. However, there was no striking disadvantageous effect beyond the influence of their relative frequency of occurrence in the dataset.

Exploring gender biases in ML and AI academic research through systematic literature review

“…An interesting paper by Crockett et al explored the gender bias effect on deception detection systems that uses Non-Verbal Behavioral cues exhibited by people and predicts if the subject is deceptive. Although they didn't find any significant effect of subject's gender on the prediction accuracy, they argued that the classifiers used to detect deception should be trained separately for different genders because that tends to work better for either gender than a one-size fits all approach (Crockett et al, 2020 ).…”

“…Crockett et al utilize raw video data collected from 32 participants to test if there is a statistically significant gender effect on the deception detection system. Through this, they find a gender effect in NVB cues generated by people, which means we cannot use a system trained on female data to detect deception on male participants and vice-versa (Crockett et al, 2020 ). The paper by Wang et al also conducts a case study into career recommender systems and implementation of debiasing technique (Wang et al, 2021 ).…”

Deception detection with machine learning: A systematic review and statistical analysis

“…Our goal is to present an abstract notion of the state of each of those dimensions so we could give researchers an insight about each theme. We hope that the following sections will [38] Neural Network / Accuracy: 1.0 [2018] Deception detection using artificial neural network and support vector machine [39] SVM / Accuracy: 1.0 [2020] Automated Deception Detection of Males and Females from Non-Verbal Facial Micro-Gestures [40] Random Forest / Accuracy: 0.998 [2019] Face-Focused Cross-Stream Network for Deception Detection in Videos [41] Neural Network / Area Under the Curve: 0.9978 [2018] A Multi-View Learning Approach To Deception Detection [42] Multi-view Learning / Accuracy: 0.98 [2015] A comparison of features for automatic deception detection in synchronous computer-mediated communication [43] Decision Tree / Accuracy: 0.98 [2019] Robust Algorithm for Multimodal Deception Detection [44] Combined methods / Accuracy: 0.97 [2018] Lie Detector with The Analysis Of The Change Of Diameter Pupil and The Eye Movement Use Method Gabor Wavelet Transform and Decision Tree [45] Decision Tree / Precision: 0.97 [2021] LieNet: A Deep Convolution Neural Networks Framework for Detecting Deception [46] Neural Network / Accuracy: 0.967375 [2017] Deep Learning Driven Multimodal Fusion for Automated Deception Detection [47] Neural Network / Accuracy: 0.964 [2019] How smart your smartphone is in lie detection? [48] KNN / Precision: 0.95 [2012] The Voice and Eye Gaze Behavior of an Imposter: Automated Interviewing and Detection for Rapid Screening at the Border [49] Decision Tree / Accuracy: 0.9447 [2020] Building a Better Lie Detector with BERT: The Difference Between Truth and Lies [50] Neural Network / Accuracy: 0.936 [2021] Deception detection in text and its relation to the cultural dimension of individualism/collectivism [51] Logistic Regression / Recall: 0.93 [2018] Deception detection in videos [52] Logistic Regression / Area Under the Curve: 0.9221 [2021] Development of Spectral Speech Features for Deception Detection Using Neural Netwo...…”

“…The performances of MLP models were measured by accuracy in 11 [39,40,43,56,66,67,71,72,83,89,102] out of the 12 studies. Those accuracy rates range from 0.6333 to 0.9665, with a mean at 0.7961 ± 0.1130.…”

“…Among the 20 studies that exploited Random Forest, 16 [40,48,56,60,66,73,75,77,78,84,86,89,92,98,105,114] measured their performance by accuracy, which ranges from 0.5677 to 0.9980, with mean at 0.7301 ± 0.1182. Three other studies [52,57,87] measured their performance by Area Under the Curve, which values 0.7330, 0.8131, and 0.8970.…”