Slower is not always better: Response-time evidence clarifies the limited role of miserly information processing in the Cognitive Reflection Test

Abstract

We report a study examining the role of ‘cognitive miserliness’ as a determinant of poor performance on the standard three-item Cognitive Reflection Test (CRT). The cognitive miserliness hypothesis proposes that people often respond incorrectly on CRT items because of an unwillingness to go beyond default, heuristic processing and invest time and effort in analytic, reflective processing. Our analysis (N = 391) focused on people’s response times to CRT items to determine whether predicted associations are evident between miserly thinking and the generation of incorrect, intuitive answers. Evidence indicated only a weak correlation between CRT response times and accuracy. Item-level analyses also failed to demonstrate predicted response-time differences between correct analytic and incorrect intuitive answers for two of the three CRT items. We question whether participants who give incorrect intuitive answers on the CRT can legitimately be termed cognitive misers and whether the three CRT items measure the same general construct.

The Smart System 1: evidence for the intuitive nature of correct responding on the bat-and-ball problem

“…Other potential support comes from latency studies. A number of studies reported that correct "5 cents" responses take considerably longer than incorrect "10 cents" responses (e.g., Alós-Ferrer, Garagnani, & Hügelschäfer, 2016;Johnson, Tubau, & De Neys, 2016;Stupple, Pitchford, Ball, Hunt, & Steel, 2017;Travers, Rolison, & Feeney, 2016). For example, in one of our own studies we observed that correct responders needed on average about a minute and a half to enter their response whereas incorrect responders only took about 30 seconds (Johnson et al, 2016).…”

“…The problem background turned yellow 2 seconds before the deadline. Note that previous studies that adopted a free response format without time-restrictions reported average response latencies for correct answers of over 30 s (Stupple et al, 2017;Johnson et al, 2016). Hence, by all means the 10 s deadline remains challenging.…”

Not All Who Ponder Count Costs: Arithmetic Reflection Predicts Utilitarian Tendencies, but Logical Reflection Predicts both Deontological and Utilitarian Tendencies

“…However, armed with PD's greater clarity, we expected arithmetic 7 Baron and colleagues also measured dilemma judgment response-times. Given that research shows only a weak correlation between response-times and other measures of reflection (Stupple, Pitchford, Ball, Hunt, & Steel, 2017) and no significant difference in response-times between CRT takers whose first response is correct and CRT takers whose first response is incorrect (Szaszi, Szollosi, Palfi, & Aczel 2017), we did not assess reaction times in the current work.…”

Assessing the validity of inferences from scores on the cognitive reflection test

“…Other studies contradict these findings. Stupple et al () found that quicker response times were not related to intuitive responses on the CRT. Baron et al () said they “failed to find any evidence that the overriding of intuitive responses is relevant to the CRT's predictive power” (p. 279).…”