How can we measure metacognition in creative problem-solving? Standardization of the MCPS scale

Abstract

Creative problem-solving is associated with the resolution of ill-defined tasks where the goals, procedure, and evaluation criteria are not predetermined. Successful problem solvers have to set their own unique goals, develop appropriate strategies, plan individual steps, generate various possible solutions, monitor and regulate the problem-solving process, and evaluate the outcome for novelty and usefulness. The present study aims to standardize the Metacognition in Creative Problem-Solving (MCPS) scale, which captures metacognitive skills (planning, monitoring, regulation, and evaluation) in creative problem-solving. The 11-item instrument was adapted from the widely recognized Motivated Strategies for Learning Questionnaire (MSLQ). The representative sample of 350 university students was employed in the exploratory phase, and 110 working professionals were employed in the confirmatory phase. Both the confirmatory factor analysis (CFA) and the item response theory (IRT) examination yielded an excellent reliability of the MCPS scale (ω = .91, and marginal reliability = .92). Moreover, the MCPS scale was correlated with creative performance (r = .26) and absolute accuracy of on-task metacognitive monitoring (r = -.19). The relationship between the MCPS scale and creative performance was stable, even after the controlling for the original MSLQ, suggesting that the MCPS scale is both a reliable and valid instrument that measures metacognition in creative problem-solving in both educational and professional environments.

Explanations from ChatGPT: Targeted Task Improvements But No Generalized Gains in Metacognitive Abilities for Novice Language Learners

“…However, few studies have speci cally examined the affordances of conversational agents like ChatGPT for developing metacognition and self-regulation. Urban et al (2023) argue that current large language models like ChatGPT lack the capabilities to promote metacognitive awareness or engage students in truly selfdirected learning. Their experimental ndings revealed that when interacting with ChatGPT, students demonstrate poorer comprehension monitoring and evaluation of responses compared to learning from a human tutor.…”

Exploring the Influence of Student Interaction with ChatGPT on Critical Thinking, Problem Solving, and Creativity

“…While traditional teaching methods endure, the infusion of AI-driven platforms presents innovative pedagogical pathways, promising to augment students' cognitive capabilities. Leveraging empirical data and statistical analyses, this research aims to unveil the multifaceted influence of varied stakeholders' engagement with ChatGPT [3]. By examining correlations between these interactions and the enhancement of critical thinking, problem-solving, and creativity, this study seeks to elucidate the pedagogical advantages of AI-based educational tools.…”