The first direct replication on using verbal credibility assessment for the detection of deceptive intentions

Abstract

Verbal deception detection has gained momentum as a technique to tell truth‐tellers from liars. At the same time, researchers' degrees of freedom make it hard to assess the robustness of effects. Replication research can help evaluate how reproducible an effect is. We present the first replication in verbal deception research whereby ferry passengers were instructed to tell the truth or lie about their travel plans. The original study found truth‐tellers to include more specific time references in their answers. The replication study that closely mimicked the setting, procedure, materials, coding, and analyses found no lie–truth difference for specific time references. Although the power of our replication study was suboptimal (0.77), Bayesian statistics showed evidence in favor of the null hypothesis. Given the great applied consequences of verbal credibility tests, we hope this first replication attempt ignites much needed preregistered, high‐powered, multilab replication efforts.

What to do with all these Bayes factors: How to make Bayesian reports in deception research more informative

“…The reporting of Bayes factors is still quite novel within deception research, though it appears to be gaining popularity (e.g., Kleinberg, Warmelink, Arntz, & Verschuere, 2018;Leal et al, 2019). We believe this is a good thing, but it does bring with it new challenges for reviewers and editors.…”

Being accurate about accuracy in verbal deception detection

“…Early findings on such independent sample validation in verbal deception research suggest that classification algorithms are less robust against sample variations than expected [31]. This independent sample validation would at the same time provide the much needed direct replications of verbal deception studies [32](for an exception see [33]). Independent sample validation ideally implies that the classifier used is pre-registered.…”

Detecting false intentions using unanticipated questions

“…Like previous studies, we modeled detail richness by using the following LIWC categories “senses” (e.g., appear, speak), “space” (e.g., wide, under), and “time” (e.g., during, until) [24–26]. These categories respectively represent the perceptual, spatial and temporal details included in the statements.…”

“…Warmelink, Vrij, Mann and Granhag [23] showed in one experiment that true travel plans did include more spatial and temporal details than false travel plans. However, in a second experiment—using a more realistic setting—and in a direct replication [24], these differences did not emerge anymore. Two studies included automatic coding instead of human coding to investigate future travel plans.…”