Acoustic-Prosodic Indicators of Deception and Trust in Interview Dialogues

Abstract

We analyze a set of acoustic-prosodic features in both truthful and deceptive responses to interview questions, identifying differences between truthful and deceptive speech. We also study the perception of deception, identifying acousticprosodic characteristics of speech that is perceived as truthful or deceptive by interviewers. In addition to studying differences across all speakers, we identify variations in deception production and perception across gender and native language. We conduct machine learning classification experiments aimed at distinguishing between truthful and deceptive speech, using acoustic-prosodic features. We also explore methods of leveraging individual traits for deception classification. Our results show that acoustic-prosodic features are highly effective at classifying deceptive speech. Our best classifier achieved an F1score of 72.77, well above both the random baseline and above human performance at this task. This work advances our understanding of deception production and perception, and has implications for automatic deception detection and the development of synthesized speech that is trustworthy.

Privacy Implications of Voice and Speech Analysis – Information Disclosure by Inference

“…While there is a rich and growing body of research to support the above statement, it has to be acknowledged that many of the studies cited in this paper achieved their classification results under ideal laboratory conditions (e.g., scripted speech, high quality microphones, close-capture recordings, no background noise) [10,20,30,36,55,60,70,82,94,107], which may raise doubt about the generalizability of their inference methods. Also, while impressive accuracies have been reached, it should not be neglected that nearly all of the mentioned approaches still exhibit considerable error rates.…”

“…Research has shown that the veracity of verbal statements can be assessed automatically [60,107]. Among other speech cues, acoustic-prosodic features (e.g., formant frequencies, speech intensity) and lexical features (e.g., verb tense, use of negative emotion words) were found to be predictive of deceptive utterances [67].…”

“…Speech-based lie detection methods have become effective, surpassing human performance [60] and almost reaching the accuracy of methods based on brain activity monitoring [107]. There is potential to further improve the classification performance by incorporating information on the speaker's personality [2], some of which can be inferred from voice recordings as well (as we have discussed in Sect.…”

Acoustic Indicators of Deception in Mandarin Daily Conversations Recorded from an Interactive Game

“…Thus, all the SUs are stored in different files under different folders. Two kinds of labeling were proposed in [2,18], one is local deception and the other is global deception. It is known that not all of the components are deceptive when a lie is told.…”

“…We follow the work in [18] to figure out the acoustic indicators. All of the p values here are corrected by controlling the false discovery rate (FDR) at α = 0.05.…”