The Cognitive Reflection Test: A Measure of Intuition/Reflection, Numeracy, and Insight Problem Solving, and the Implications for Understanding Real-World Judgments and Beliefs

Abstract

The Cognitive Reflection Test (CRT) has increasingly dominated theorizing about individual differences in intuitive/reflective thinking propensities, and it is associated with many real-world beliefs and judgments, such as religiosity, paranormal beliefs, and moral judgments. The CRT triggers common incorrect responses that come to mind easily, and it is assumed that recognizing this error is tantamount to solving the problems. As a result, incorrect answers on the CRT purportedly indicate an intuitive thought process, whereas correct answers purportedly indicate a reflective thought process. It has also been argued that the CRT problems are fundamentally different from insight problems because insight problems often cause people to sit lost in thought, unable to solve the problem until they correctly reframe it. The present research tested these assumptions and found that a substantial proportion of people have difficulty solving the problems even when the “intuitive” response is unavailable to them, the correct answer is among four multiple-choice options, and they take time to reflect. Associations between the CRT and beliefs (religiosity, paranormal beliefs, moral judgments, etc.) remained even under conditions in which CRT errors appeared to result from more reflective thought than correct responses. Furthermore, multi-dimensional item response theory models indicated that the CRT loaded onto numeracy and insight problem solving ability factors rather than its own unique factor. Additionally, regression analyses indicated that numeracy and insight may account for many associations between the CRT and real-world beliefs. Broader implications for dual-process theories of reasoning and judgment are discussed.

Measuring cognitive reflection without maths: Development and validation of the verbal cognitive reflection test

“…Because the CRT-V is less confounded by numeracy-that accounts for unique variance in paranormal beliefs (Patel, 2017;Sirota & Juanchich, 2018)-we expected the correlation for the CRT-V to be attenuated (e.g., the correlation between the numerical CRT and paranormal beliefs dropped from −27 to −18 when controlling for numeracy, Sirota & Juanchich, 2018).…”

Not All Who Ponder Count Costs: Arithmetic Reflection Predicts Utilitarian Tendencies, but Logical Reflection Predicts both Deontological and Utilitarian Tendencies

“…Because the CRT involves comprehending and processing mathematical information, some researchers have worried that the CRT is just another numeracy test (Campitelli & Labollita, 2010). However, subsequent work indicates that while CRT performance does correlate with numeracy, the CRT is not reducible to a numeracy test (Liberali, Reyna, Furlan, Stein, & Pardo, 2012;Patel, 2017;Primi, Morsanyi, Chiesi, Donati, & Hamilton, 2016)-it also measures the inhibition of default responses (e.g., Campitelli & Gerrans, 2014), miserly reasoning (Toplak, West, & Stanovich, 2011), and actively open-minded thinking (Szaszi, Szollosi, Palfi, & Aczel, 2017).…”

Not Just Asking Questions: Effects of Implicit and Explicit Conspiracy Information About Vaccines and Genetic Modification

“…Following Pennycook and Rand (2018), cognitive reflection (M = 0.52, SD = .30) was measured by combining the three-item standard Cognitive Reflection Test (CRT) using alternate wording (Patel, 2017), and the 4-item non-numeric CRT-2 (Thomson & Oppenheimer, 2016). Both were presented in multiple choice format (Patel, 2017), with the intuitive-incorrect choice listed first. Correct responses were scored as 1, and all other responses as 0.…”