Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules

Abstract

Research on judgment and decision making has suggested that the System 2 process of slow thinking can help people to improve their decision making by reducing well-established statistical decision biases (including base rate neglect, probability matching, and the conjunction fallacy). In a large pre-registered study with 1,706 participants and 23,292 unique observations, we compare the effects of individual differences and behavioral interventions to test the relative benefits of slow thinking on performance in canonical judgment and decision-making problems, compared to a control condition, a fast thinking condition, an incentive condition, and a condition that combines fast and slow thinking. We also draw on the rule-based reasoning literature to examine the benefits of having access to a simple form of the rule needed to solve a specific focal problem. Overall, we find equivocal evidence of a small benefit from slow thinking, evidence for a small benefit to accuracy incentives, and clear evidence of a larger cost from fast thinking. The difference in performance between fast-thinking and slow-thinking interventions is comparable to a one-scale point difference on the 4-point Cognitive Reflection Test (CRT). Inferential rules contribute unique explanatory power and interact with individual differences to support the idea that System 2 benefits from a combination of slower processes and knowledge appropriate to the problem.

Activating reflective thinking with decision justification and debiasing training

“…Similarly, studies that rely on the tworesponse paradigm, where an initial (relatively more intuitive) response is elicited before a second (relatively less intuitive and more reflected) response, often lack a control condition (e.g., Bago & De Neys, 2017). As a recent exception, Lawson, Larrick, and Soll (2020) employ slow and fast thinking prompts (without time-limits) and find that slow thinking has limited positive effect on cognitive performance compared to a control condition. Given its importance, we also employ control conditions in the current study.…”

“…Studies using intuition and reflection manipulations often do not directly test whether cognitive processes were activated in the intended directions. While some have checked the direct effects of their manipulations on cognitive performance (e.g., Deppe et al, 2015;Lawson et al, 2020;Yilmaz & Saribay, 2016), subjective self-report questions and behavioral measures such as response times are frequently relied on as alternative manipulation checks Yilmaz & Isler, 2019). The lack of performance measures would be misleading if, rather than thinking reflectively about the problem at hand, participants were to rely on their own lay theories about reflection (Saribay, Yilmaz & Körpe, 2020) or if they were to respond in socially desirable ways (Grimm, 2010).…”

“…Therefore, the effect of reflection manipulations should be observed on well-established measures of cognitive performance -such as the CRT (Frederick, 2005) and the CRT-2 (Thomson & Oppenheimer, 2016). Providing evidence of their ability to predict the domain-general features of reflection, test scores on these two tasks have been shown to correlate with a wide-range of cognitive performance measures in the lab (e.g., syllogistic reasoning and heuristics-and-biases problems) and in the field (e.g., standardized academic test scores and university course grades) (Lawson et al, 2020;Meyer, Zhou & Shane, 2018;Thomson & Oppenheimer, 2016;Toplak, West, & Stanovich, 2011). Numerous other widely-used reasoning problems, such as the conjunction fallacy (Tversky & Kahneman, 1983), probability matching (Stanovich & West, 2008) and base rate neglect (Kahneman & Tversky, 1973), can also be used to measure the effects of manipulations on cognitive performance (e.g., Lawson et al, 2020).…”

“…Providing evidence of their ability to predict the domain-general features of reflection, test scores on these two tasks have been shown to correlate with a wide-range of cognitive performance measures in the lab (e.g., syllogistic reasoning and heuristics-and-biases problems) and in the field (e.g., standardized academic test scores and university course grades) (Lawson et al, 2020;Meyer, Zhou & Shane, 2018;Thomson & Oppenheimer, 2016;Toplak, West, & Stanovich, 2011). Numerous other widely-used reasoning problems, such as the conjunction fallacy (Tversky & Kahneman, 1983), probability matching (Stanovich & West, 2008) and base rate neglect (Kahneman & Tversky, 1973), can also be used to measure the effects of manipulations on cognitive performance (e.g., Lawson et al, 2020). Among these alternatives, we chose CRT-2 as our performance measure because participants are less likely to be familiar with it, thereby minimizing problems such as ceiling effects, and because its reliance on numeracy skills is less than that of CRT, which can confound the interpretation of scores (see discussion in Thomson & Oppenheimer, 2016).…”

National Narcissism predicts the Belief in and the Dissemination of Conspiracy Theories During the COVID-19 Pandemic: Evidence From 56 Countries

“… 8. There is a debate in the literature on what construct(s) such tests capture (e.g., analytical processing, numeracy, executive control, disposition to be careful and reflective; see Lawson et al, 2020 and Szaszi et al, 2017 ). …”