Automated verbal credibility assessment of intentions: The model statement technique and predictive modeling

Abstract

SummaryRecently, verbal credibility assessment has been extended to the detection of deceptive intentions, the use of a model statement, and predictive modeling. The current investigation combines these 3 elements to detect deceptive intentions on a large scale. Participants read a model statement and wrote a truthful or deceptive statement about their planned weekend activities (Experiment 1). With the use of linguistic features for machine learning, more than 80% of the participants were classified correctly. Exploratory analyses suggested that liars included more person and location references than truth‐tellers. Experiment 2 examined whether these findings replicated on independent‐sample data. The classification accuracies remained well above chance level but dropped to 63%. Experiment 2 corroborated the finding that liars' statements are richer in location and person references than truth‐tellers' statements. Together, these findings suggest that liars may over‐prepare their statements. Predictive modeling shows promise as an automated veracity assessment approach but needs validation on independent data.

‘Language of lies’: Urgent issues and prospects in verbal lie detection research

“…Specifically, we propose several means to improve the coding of statements for verbal credibility assessment: Laboratories should specify their coding scheme prior to data collection and make them available to others. These schemes should be accompanied with coded example statements (explaining the coding) and exercise statements that allow others to adopt the coding scheme and assess their coding skill. Laboratories should collaborate to examine the reliability (and validity) of different coding schemes, preferable on openly available data sets (Kleinberg, Nahari, Arntz, & Verschuere, ; Kleinberg, van der Toolen, Vrij, Arntz, & Verschuere, ; Mihalcea, Narvaez, & Burzo, ; Ott, Choi, Cardie, & Hancock, ; Ott et al ., ; Pérez‐Rosas, Abouelenien, Mihalcea, & Burzo, ). Perfectly reliable, automated scoring might currently lack an understanding of contextual information. We should, however, explore whether verbal coding can be operationalized as a joint effort between computer and humans (e.g., human‐in‐the‐loop where computer codes for detail, and a human coder makes adjustments based on well‐specified contextual considerations). …”

“…2. Laboratories should collaborate to examine the reliability (and validity) of different coding schemes, preferable on openly available data sets (Kleinberg, Nahari, Arntz, & Verschuere, 2017;Kleinberg, van der Toolen, Vrij, Arntz, & Verschuere, 2018;Mihalcea, Narvaez, & Burzo, 2014;Ott, Choi, Cardie, & Hancock, 2011;Ott et al, 2013; P erez-Rosas, Abouelenien, Mihalcea, & Burzo, 2015). 3.…”

Unexpected questions in deception detection interviews: Does question order matter?

“…They create an answer that is more detailed than most truth tellers are willing or able to give about their intentions. A similar ‘over‐preparation’ effect was found in written truthful or deceptive statements about intentions (Kleinberg, van der Toolen, Vrij, Arntz, & Verschuere, ).…”

“…Asking unexpected questions first might disrupt liars' strategies to be as convincing as possible and lead to better deception detection. Vrij et al (2018) studied similar order effects in interpreter-mediated interviews that included questions about a trip (expected) and questions about the planning of a trip (unexpected). They found that the order of these questions did not affect the amount of detail provided.…”

The first direct replication on using verbal credibility assessment for the detection of deceptive intentions

“…Previously, it has been argued that expected, factual questions are what liars prepare for and can, therefore, enrich with details (Warmelink et al, 2012). In support of that idea, people who lied about their planned weekend activities mentioned more persons and more locations than those who told the truth (Kleinberg, van der Toolen, Vrij, Arntz, & Verschuere, 2018). A working hypothesis states that liars might overcompensate in their statements because they are particularly inclined to appear convincing whereas truth‐tellers assume that their truth will appear naturally.…”