Automated feedback and creativity: On the role of metacognitive monitoring in divergent thinking.

Abstract

Creativity can be seen as the interplay of various cognitive factors, such as associative, executive, and metacognitive processes. Here, we test the specific role of metacognitive monitoring and how external feedback can support creative ideation. In a series of three studies, we show the benefits of a novel automated technique that uses semantic analysis to assess originality and provides real-time feedback as ideas are generated. In two divergent thinking tasks (alternate uses tests), participants who received feedback outperformed those who received no feedback (Study 1, n = 120, preregistered), random feedback (Study 2, n = 250), or repeated be-creative instructions (Study 3, n = 264). In all studies, an appreciation scale showed that the feedback was perceived as helpful, valid, motivating, and clear. Random feedback was found to be less valid, supporting the validity of the original intervention. Self-reported levels of positive and negative emotions were examined as a competing explanation for the feedback effect. Receiving feedback did not result in different levels of emotions compared to receiving no feedback or be-creative instructions. However, an interaction showed that higher levels of positive emotion were associated with greater originality when receiving feedback. Random feedback resulted in lower levels of positive emotion, but no interaction. These studies highlight the role of metacognition in divergent thinking, as well as the applied potential of semantic analysis and the use of technology-assisted feedback interventions for creativity. The role of possible emotional and motivational factors, as well as limitations in the use of semantic analysis, is discussed.

How do examples impact divergent thinking? The interplay between associative and executive processes.

“…Automated scoring systems show good convergent validity with human raters (Beaty & Johnson, 2021; Beaty et al, 2022; Dumas et al, 2021) and the Torrance Test of Creative Thinking norms (Acar et al, 2023). The average intraclass correlation of OCS scores with human raters was reported to be 0.71 ( SD = 0.11) in eight studies with open-access data (de Chantal & Organisciak, 2023). It should be noted that automated systems have been shown to be reliable in evaluating originality, but not usefulness (e.g., Beaty & Johnson, 2021).…”

Measuring original thinking in elementary school: Development and validation of a computational psychometric approach.

“…By becoming an integral part of creativity development efforts in schools, MOTES aligns with the objective of enhancing higher-order thinking skills and 21st-century competencies. Notably, a recent study by de de Chantal and Organisciak (2023) employed a similar automated scoring framework as MOTES to provide instant feedback for enhancing the creativity of responses. This is a very promising progress because it facilitates scaffolding for creativity (Hathcock et al, 2015) thanks to the instant nature of the feedback provided by automated scores.…”

Contributions of Metacognition to Creative Performance and Behavior

“…Thus, the knowledge about cognition supports effective metacognitive monitoring. People who receive feedback during idea generation perform better and generate more original ideas earlier in the process (De Chantal & Organisciak, 2023). It can be assumed that increases in creativity-related knowledge (e.g., learning about what ideas can be considered more vs less creative) may help to better calibrate one's metacognitive monitoring.…”

“…Based on the concept of procedural metacognition, we assume that the three components of creative metacognition described above dynamically interact (De Chantal & Organisciak, 2023;Nelson & Narens, 1994). Research indicates that based on metacognitive feelings of low fluency resulting from a decreasing number of generated ideas over time, people develop the misconception that their creative performance declines, what leads them to prematurely stop of generating ideas.…”