Multimodal Political Deception Detection

Kamboj, Manvi; Hessler, C; Asnani, Priyanka; Riani, Kais; Abouelenien, Mohamed

doi:10.1109/mmul.2020.3048044

Cited by 13 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We kept Chinese and Mandarin apart because they were referred to as such. Furthermore, most studies on the aforementioned languages are devoted to vocal cues [ 38 , 63 , 68 , 70 , 75 , 80 – 82 , 88 , 97 , 105 , 111 , 114 , 115 ].…”

Section: Discussionmentioning

confidence: 99%

Deception detection with machine learning: A systematic review and statistical analysis

et al. 2023

View full text Add to dashboard Cite

Several studies applying Machine Learning to deception detection have been published in the last decade. A rich and complex set of settings, approaches, theories, and results is now available. Therefore, one may find it difficult to identify trends, successful paths, gaps, and opportunities for contribution. The present literature review aims to provide the state of research regarding deception detection with Machine Learning. We followed the PRISMA protocol and retrieved 648 articles from ACM Digital Library, IEEE Xplore, Scopus, and Web of Science. 540 of them were screened (108 were duplicates). A final corpus of 81 documents has been summarized as mind maps. Metadata was extracted and has been encoded as Python dictionaries to support a statistical analysis scripted in Python programming language, and available as a collection of Jupyter Lab Notebooks in a GitHub repository. All are available as Jupyter Lab Notebooks. Neural Networks, Support Vector Machines, Random Forest, Decision Tree and K-nearest Neighbor are the five most explored techniques. The studies report a detection performance ranging from 51% to 100%, with 19 works reaching accuracy rate above 0.9. Monomodal, Bimodal, and Multimodal approaches were exploited and achieved various accuracy levels for detection. Bimodal and Multimodal approaches have become a trend over Monomodal ones, although there are high-performance examples of the latter. Studies that exploit language and linguistic features, 75% are dedicated to English. The findings include observations of the following: language and culture, emotional features, psychological traits, cognitive load, facial cues, complexity, performance, and Machine Learning topics. We also present a dataset benchmark. Main conclusions are that labeled datasets from real-life data are scarce. Also, there is still room for new approaches for deception detection with Machine Learning, especially if focused on languages and cultures other than English-based. Further research would greatly contribute by providing new labeled and multimodal datasets for deception detection, both for English and other languages.

show abstract

Section: Discussionmentioning

confidence: 99%

Deception detection with machine learning: A systematic review and statistical analysis

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In addition, used classifier algorithm could play a big role on the outcome. This is evident by the research conducted by Kamboj et al [27] and Şen et al [31], where the low performance of the first one was rationalized by the authors to be due to acoustic features having inherently low discriminating power. Meanwhile, the second work had its highest performance when combining acoustic features with the facial features.…”

Section: Audio Based Deception Detectionmentioning

confidence: 95%

“…As previously discussed, attempting a multimodal approach can significantly improve the classification accuracy of DD; however, it isn't clear that it guarantees improvement to the classification performance. For instance, a recent study by Kamboj et al [27] has achieved an accuracy of 70% using a combination of lexical, acoustic and visual features, or in the case of Şen et al [31], 72% accuracy was achieved when combining all visual, acoustic and linguistic features as opposed to 84.18% with only visual and acoustic modals. This may be attributed to some modals having less discriminative power compared to others depending on the approach, and methodology.…”

Section: Audio Based Deception Detectionmentioning

confidence: 99%

A Review on Non-Invasive Multimodal Approaches to Detect Deception Based on Machine Learning Techniques

Baraa M. Albaker

2024

jes

View full text Add to dashboard Cite

Detecting deception has been investigated by the scientific community for over a century due to its importance in the justice system and homeland security. Attempts to come up with an approach, a system or a framework that serves the purpose of discerning lies from truths has therefore been a major field. This has led researchers to automate the detection process and reduce its invasiveness as much as possible. In addition, machine learning techniques are used with multiple channels of information, known as modals, to increase accuracy in what is known as a multimodal approach. As a result, several research and datasets are currently available, and it could be challenging to identify successful patterns, gaps, and future directions. In this paper, over fifty state-of-the-art publications in the field of deception detection using non-invasive approaches based on machine learning techniques are analyzed after reviewing more than one thousand publications from Scopus, IEEE Xplore, Web of Science, ScienceDirect, and Google Scholar. The work presents the classification techniques and datasets used with their detection performance and finally analyzing the data to draw conclusions. The reported detection accuracy ranges from about 50% to 95% for monomodal approaches based on facial expression, body movement, audio, or thermal imaging. In conclusion, the multimodal approach shows promising results as it reaches a detection accuracy approaching 100%. It outperforms any alternative non-invasive approach, especially when dealing with small datasets, which seems to be the biggest challenge in this field. Future research directions should focus on experimenting with multimodal systems by developing larger datasets as well as implementing classification algorithms that can work with multiple modals effectively.

show abstract

“…They achieved an accuracy of up to 98%. Kamboj et al [12] attempted to create their own dataset by collecting videos from the internet of political gures and labeling them using PolitiFact. The target modals they were aiming for are linguistic, acoustic, and visual features.…”

Section: Related Workmentioning

confidence: 99%

Non-Invasive Real-Time Multimodal Deception Detection Using Machine Learning and Parallel Computing Techniques

Abdulridha,

Albaker

2024

Preprint

View full text Add to dashboard Cite

Detecting deception reliably has been a sought-after goal for researchers since the early 20th century. This is due to its high stakes nature, especially when considering the setting under which a deception detection system would be utilized, such as law enforcement sectors, judicial bodies, and criminal investigation bodies. Therefore, recent literature has greatly focused on developing systems with ever increasing accuracies to minimize false positives. Attempts to diversify the sources of data being analyzed and classified as well as the reliance on artificial intelligence technologies have shown great success. But little attention was paid to the applicability of these systems in real-life scenarios. an idle deception detection system needs to exhibit accuracy but also perform in real-time, a feature that is lacking in the current state-of-the-art. In this work, a non-invasive real-time multimodal deception detection system is developed using advanced machine learning techniques. It combines data sources from video and audio streams to extract visual, acoustic, and linguistic features. It also utilizes parallel computing techniques to ensure high performance adequate for real-time usage. Furthermore, a user-friendly graphical user interface was built to facilitate the use of the system. Multiple experiments were conducted to determine its accuracy under various circumstances and combinations of features, the system had a detection accuracy of 99.83% under real-life, high-stakes scenarios using visual and acoustic features, and 91.03% accuracy under controlled environments, while an 89.54% accuracy was achieved using mixed environments and using visual, acoustic, and linguistic features.

show abstract

Multimodal Political Deception Detection

Cited by 13 publications

References 14 publications

Deception detection with machine learning: A systematic review and statistical analysis

Deception detection with machine learning: A systematic review and statistical analysis

A Review on Non-Invasive Multimodal Approaches to Detect Deception Based on Machine Learning Techniques

Non-Invasive Real-Time Multimodal Deception Detection Using Machine Learning and Parallel Computing Techniques

Contact Info

Product

Resources

About