The truth revisited: Bayesian analysis of individual differences in the truth effect

Abstract

The repetition-induced truth effect refers to a phenomenon where people rate repeated statements as more likely true than novel statements. In this paper, we document qualitative individual differences in the effect. While the overwhelming majority of participants display the usual positive truth effect, a minority are the opposite—they reliably discount the validity of repeated statements, what we refer to as negative truth effect. We examine eight truth-effect data sets where individual-level data are curated. These sets are composed of 1105 individuals performing 38,904 judgments. Through Bayes factor model comparison, we show that reliable negative truth effects occur in five of the eight data sets. The negative truth effect is informative because it seems unreasonable that the mechanisms mediating the positive truth effect are the same that lead to a discounting of repeated statements’ validity. Moreover, the presence of qualitative differences motivates a different type of analysis of individual differences based on ordinal (i.e., Which sign does the effect have?) rather than metric measures. To our knowledge, this paper reports the first such reliable qualitative differences in a cognitive task.

Are There Reliable Qualitative Individual Difference in Cognition?

“…This is a bias of sorts, and it probably is used on statements where the participant is guessing about the truth of the statement. Schnuerch et al (2020) found that wherever individual differences were resolvable the unconstrained model outperformed the positive model. Their conclusion was that while most people had a true truth effect, a minority had the opposite.…”

“…Here is a less obvious example. Schnuerch, Nadarevic, & Rouder (2020) examined individual differences in the truth effect with the methods outlined here. The truth effect occurs when people rate statements as more likely to be true when they are repeated.…”

“…This minority truly discounted the truth of repeated statements. And Schnuerch et al (2020) further concluded that it was unlikely that the mental processes that gave rise to the truth effect were the same as those that gave rise to its reversal. Hence, the effect of repetition was complex and varied, and most likely dependent on additional factors such as how conscious people are of previously encountering the item (Nadarevic & Erdfelder, 2014).…”

Quantitative and Qualitative Differences in the Canonical and the Reverse Distance Effect and Their Selective Association With Arithmetic and Mathematical Competencies

“…Based on this classification we calculated the percentage of individuals who showed a "positive", a "negative" or no (undecided) distance effect. The benchmark for choosing a positive/negative classification was based on a similar classification by Schnuerch et al (2020). Please note that a posterior probability of 0.75 equates to an odds ratio of 3-to-1, which is considered a minimum threshold of evidence in Bayesian model comparison.…”

“…The best model fit is then tested using a Bayes factor model comparison. This novel approach has been successfully implemented to test the structure of individual differences in numerical priming effects (Haaf and Rouder, 2019), location and color Stroop effects (Haaf and Rouder, 2019), numerical size congruity effects (Faulkenberry and Bowman, 2020), and the truth effect (Schnuerch et al, 2020).…”