The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. In this study, we showed that the CRT is a more potent predictor of performance on a wide sample of tasks from the heuristics-and-biases literature than measures of cognitive ability, thinking dispositions, and executive functioning. Although the CRT has a substantial correlation with cognitive ability, a series of regression analyses indicated that the CRT was a unique predictor of performance on heuristics-and-biases tasks. It accounted for substantial additional variance after the other measures of individual differences had been statistically controlled. We conjecture that this is because neither intelligence tests nor measures of executive functioning assess the tendency toward miserly processing in the way that the CRT does. We argue that the CRT is a particularly potent measure of the tendency toward miserly processing because it is a performance measure rather than a self-report measure.

Background: Both performance-based and rating measures are commonly used to index executive function in clinical and neuropsychological assessments. They are intended to index the same broad underlying mental construct of executive function. The association between these two types of measures was investigated in the current article. Method and Results: We examined the association between performance-based and rating measures of executive function in 20 studies. These studies included 13 child and 7 adult samples, which were derived from 7 clinical, 2 nonclinical, and 11 combined clinical and nonclinical samples. Only 68 (24%) of the 286 relevant correlations reported in these studies were statistically significant, and the overall median correlation was only .19. Conclusions: It was concluded that performance-based and rating measures of executive function assess different underlying mental constructs. We discuss how these two types of measures appear to capture different levels of cognition, namely, the efficiency of cognitive abilities and success in goal pursuit. Clinical implications of using performance-based and rating measures of executive function are discussed, including the use of these measures in assessing ADHD.

No abstract

Performance-based measures and ratings of executive functions were examined in a sample of adolescents with attention deficit/hyperactivity disorder (ADHD) and comparison controls. Performance-based measures of executive function included inhibition, working memory, set shifting, and planning, and ratings of these same executive functions were completed by parents and teachers. Adolescents with ADHD demonstrated lower executive function performance than controls and displayed elevated ratings on the executive function ratings by parents and teachers. Significant associations were obtained between the performance-based measures and the parent and teacher ratings, but each measure was not uniquely associated with its respective scale on the rating scales. When performance-based measures and ratings were examined as predictors of ADHD status, the parent and teacher ratings entered as significant predictors of ADHD status. Further commonality analyses indicated that performance-based measures accounted for little unique variance in predicting ADHD status and also displayed little overlap with the behavioral ratings. These findings highlight the diagnostic utility of behavioral ratings of executive function in predicting ADHD status; however, behavioral ratings should not be assumed to be a proxy for performance on measures of executive function in clinical practice.

In this article, the authors argue that there are a range of effects usually studied within cognitive psychology that are legitimately thought of as aspects of critical thinking: the cognitive biases studied in the heuristics and biases literature. In a study of 793 student participants, the authors found that the ability to avoid these biases was moderately correlated with a more traditional laboratory measure of critical thinking-the ability to reason logically when logic conflicts with prior belief. The correlation between these two classes of critical thinking skills was not due to a joint connection with general cognitive ability because it remained statistically significant after the variance due to cognitive ability was partialed out. Measures of thinking dispositions (actively open-minded thinking and need for cognition) predicted unique variance in both classes of critical thinking skills after general cognitive ability had been controlled.