Box of Lies: Multimodal Deception Detection in Dialogues

Soldner, Felix; Pérez-Rosas, Verónica; Mihalcea, Rada

doi:10.18653/v1/n19-1175

Cited by 34 publications

(30 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparison, single models of Cabtoost, XGBoost and LightGBM have achieved results in terms of F-score of 84.1%, 84.6%, and 85.0% respectively. The achieved empirical results are highly competitive and comparable with the results presented by other researchers [11,12,17,18,21,20,23] (see Table 1). We present our results achieved with the use of two corpora and the approach described above.…”

Section: Discussion Of the Resultssupporting

confidence: 86%

“…Approach from [11] F-score = 63.9%, Precision = 76.1% Approach from [12] UAR = 74.9% Baseline system [17] UAR = 68.3% Approach from [18] Accuracy (max) = 75.0% Approach from [21] UAR (max) = 70.0% Approach from [20] UAR = 73.5%, F-score = 75.0%, Precision = 77.0% Approach from [23] Accuracy…”

Section: Approach Classification Resultsmentioning

confidence: 99%

“…They used classifiers (Decision Trees and Random Forest) both for verbal and non-verbal features and achieved Accuracy in range of 60.0-75.0%. The same authors later presented another multimodal corpus that included deceptive and truthful samples [18]. The paper also proposed a model of Random Forest that achieved Accuracy = 69.0%.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Detection of Deceptive and Truthful Paralinguistic Information in Speech using Two-Level Machine Learning Model

Velichko¹,

Saint-Petersburg²,

Karpov³

2021

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

In this work, we present a novel approach to one of computational paralinguistic tasks -automatic detection of deceptive and truthful information in human's speech. This task belongs to the aspects of destructive behaviour and was first presented at the International INTERSPEECH Computational Paralinguistics Challenge ComParE in 2016. The need of contactless method for deception detection follows from the fact that existing contact-based approaches such as polygraphs and lie detectors have multiple restrictions, which significantly limit their usage. Both for training and testing of the proposed models we used two English-language corpora (Deceptive Speech Database and Real-Life Trial Deception Detection Dataset). We extracted tree sets of acoustic features from those audio samples using openSMILE toolkit. The proposed approach includes preprocessing of the extracted acoustic features with the usage of methods for data augmentation and dimensionality reduction of feature space. We have got 1680 speech utterances and 986-dimensional informative feature vector for each utterance. The main part of the proposed approach is two-level recognition model, where the first level includes three models of gradient boosting (Catboost, XGBoost and LightGBM). The second level consists of logistic regression-based model for final prediction on truthfulness or deceptiveness that takes into account predictions from the first level. Using this approach, we have achieved the result of classification in terms of F-score = 85.6%. The proposed approach can be used both independently and as a component of multimodal systems for detection of deceptive and truthful utterances in speech, as well as in systems for detection of a destructive behaviour.

show abstract

Section: Discussion Of the Resultssupporting

confidence: 86%

Section: Approach Classification Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Detection of Deceptive and Truthful Paralinguistic Information in Speech using Two-Level Machine Learning Model

Velichko¹,

Saint-Petersburg²,

Karpov³

2021

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

show abstract

“…The use of several modalities for lie detection was also investigated to see its impact in improving detection algorithms. In [30][31][32], both verbal and non-verbal features were utilized. The verbal features were extracted from linguistic features in transcriptions, while non-verbal ones consisted of binary features containing information about facial and hands gestures.…”

Section: Related Workmentioning

confidence: 99%

“…The verbal features were extracted from linguistic features in transcriptions, while non-verbal ones consisted of binary features containing information about facial and hands gestures. In addition, Soldner et al [32] introduced dialogue features, consisting of interaction cues. Other multi-modal approaches combined the previously mention verbal and non-verbal features together with micro-expressions [3][4][5], thermal imaging [33], or spatio-temporal features extracted from 3D CNNs [34,35].…”

Section: Related Workmentioning

confidence: 99%

Machine Learning-Based Lie Detector Applied to a Novel Annotated Game Dataset

et al. 2021

View full text Add to dashboard Cite

Lie detection is considered a concern for everyone in their day-to-day life, given its impact on human interactions. Thus, people normally pay attention to both what their interlocutors are saying and to their visual appearance, including the face, to find any signs that indicate whether or not the person is telling the truth. While automatic lie detection may help us to understand these lying characteristics, current systems are still fairly limited, partly due to lack of adequate datasets to evaluate their performance in realistic scenarios. In this work, we collect an annotated dataset of facial images, comprising both 2D and 3D information of several participants during a card game that encourages players to lie. Using our collected dataset, we evaluate several types of machine learning-based lie detectors in terms of their generalization, in person-specific and cross-application experiments. We first extract both handcrafted and deep learning-based features as relevant visual inputs, then pass them into multiple types of classifier to predict respective lie/non-lie labels. Subsequently, we use several metrics to judge the models’ accuracy based on the models predictions and ground truth. In our experiment, we show that models based on deep learning achieve the highest accuracy, reaching up to 57% for the generalization task and 63% when applied to detect the lie to a single participant. We further highlight the limitation of the deep learning-based lie detector when dealing with cross-application lie detection tasks. Finally, this analysis along the proposed datasets would potentially be useful not only from the perspective of computational systems perspective (e.g., improving current automatic lie prediction accuracy), but also for other relevant application fields, such as health practitioners in general medical counselings, education in academic settings or finance in the banking sector, where close inspections and understandings of the actual intentions of individuals can be very important.

show abstract