Does Interference Between Intuitive Conceptions and Scientific Concepts Produce Reliable Inter-individual Differences? A Psychometric Analysis

Abstract

Statement-verification studies indicate the coexistence of intuitive conceptions and scientific concepts within learners. The underlying assumption is that the intuitive conceptions typically built in childhood never disappear, but are co-activated with scientific concepts when we face relevant situations. This is visible in increased reaction times and error rates when confronting statements for which intuitive conceptions and scientific concepts disagree. Studies that have tried to relate individual differences in this phenomenon to other variables such as inhibition have yielded inconclusive results. In the present study, we examine the structure of individual differences in the statement-verification paradigm by Shtulman and Valcarcel (2012). Using factor analytic model comparisons, we find that there is little stable variation in individuals’ increases of reaction times and error rates across items covering scientific concepts from different domains, as well as across topics within domains. This finding suggests that a reason for inconsistent findings is the absence of stable individual differences in statement-verification tasks. One explanation for this finding is that the answer process draws more strongly on variable cognitive resources (e.g., content knowledge) than on more stable resources (e.g., inhibition ability). The occurrence and handling of interference between intuitive conceptions and scientific concepts appear to be driven by contextual factors rather than by stable cognitive processes. We discuss alternative explanations for the lack of reliable individual differences and implications for theories and the statistical modeling of cognitive pluralism.

Collaborative Problem-Solving in Knowledge-Rich Domains: A Multi-Study Structural Equation Model

“…We want to point out that the internal consistency of knowledge as a construct-determined by the intercorrelations among knowledge pieces-typically exhibits a moderate level. Importantly, recent research argues that a moderate level of internal consistency does not undermine the constructs' capacity to explain a significant amount of variance (Edelsbrunner, 2024;Stadler et al, 2021;Taber, 2018). Content knowledge was separated into radiology and internal medicine knowledge, as these two disciplines play a major role in the diagnosis of the simulated patient cases.…”

“…While we chose to use omega values as a more appropriate measure of reliability in our context, given the complex and multifaceted nature of the knowledge constructs, these lower-than-expected values raise important questions about the quality of the data and the robustness of the findings. Thus, it is important to understand that knowledge constructs, by their very nature, may not always exhibit high levels of internal consistency due to the diverse and interrelated components they encompass (Edelsbrunner, 2024;Stadler et al, 2021;Taber, 2018). This complexity may be reflected in the moderate omega values observed, which, while seemingly counterintuitive, does not invalidate the potential of the constructs to account for substantial variance in related outcomes.…”