The Dynamics of Creative Ideation: Introducing a New Assessment Paradigm

Abstract

Despite six decades of creative cognition research, measures of creative ideation have heavily relied on divergent thinking tasks, which still suffer from conceptual, design, and psychometric shortcomings. These shortcomings have greatly impeded the accurate study of creative ideation, its dynamics, development, and integration as part of a comprehensive psychological assessment. After a brief overview of the historical and current anchoring of creative ideation measurement, overlooked challenges in its most common operationalization (i.e., divergent thinking tasks framework) are discussed. They include (1) the reliance on a single stimulus as a starting point of the creative ideation process (stimulus-dependency), (2) the analysis of response quality based on a varying number of observations across test-takers (fluency-dependency), and (3) the production of “static” cumulative performance indicators. Inspired from an emerging line of work from the field of cognitive neuroscience of creativity, this paper introduces a new assessment framework referred to as “Multi-Trial Creative Ideation” (MTCI). This framework shifts the current measurement paradigm by (1) offering a variety of stimuli presented in a well-defined set of ideation “trials,” (2) reinterprets the concept of ideational fluency using a time-analysis of idea generation, and (3) captures individual dynamics in the ideation process (e.g., modeling the effort-time required to reach a response of maximal uncommonness) while controlling for stimulus-specific sources of variation. Advantages of the MTCI framework over the classic divergent thinking paradigm are discussed in light of current directions in the field of creativity research.

Automating creativity assessment with SemDis: An open platform for computing semantic distance

“…A strength of the AUT is that it seems to offer a good approximation of a person's general capacity to come up with original ideas. Although it has limitations (Barbot, 2018), the AUT and other divergent thinking tests have shown consistent evidence of validity, with several studies reporting moderate to large correlations between AUT performance and real-world creative achievement in the arts and sciences (Beaty et al, 2018;Plucker, 1999). Paul Torrance, who developed a widely used creativity assessment (the Torrance Test of Creative Thinking; TTCT), provided perhaps the most compelling longitudinal evidence for the validity of divergent thinking tests: highly creative children-assessed by performance on the TTCT-grew up to be highly creative adults, reporting significantly more creative accomplishments when assessed decades later in adulthood (Plucker, 1999;Torrance, 1981); remarkably, a 50-year follow-up of Torrance's data further confirmed the validity of divergent thinking in predicting students' future creative accomplishment (Runco, Millar, Acar, & Cramond, 2010).…”

“…Divergent thinking responses are often scored on two dimensions: fluency (the total number of responses) and originality (the creative quality of responses). Fluency offers a proxy of generative ability; however, it has been criticized for a lack of reliability, with inter-item fluency correlations on the AUT often as low as .3 to .4 (Barbot, 2018;cf. Dumas & Dunbar, 2014).…”

Automating Creativity Assessment with SemDis: An Open Platform for Computing Semantic Distance

“…Although each method has shown some degree of utility for creativity research, each comes with challenges and limitations. Two challenges that are common to most creativity assessments are subjectivity (raters don't always agree on what's creative) and labor cost (raters often have to score thousands of responses by hand)-both of which pose threats to the reliable and valid assessment of creativity (Barbot, 2018;Forthmann, Holling, Zandi, et al, 2017;Reiter-Palmon, Forthmann, & Barbot, 2019). To address these issues, researchers have begun to explore whether the process of scoring responses for their creative quality can be automated and standardized using computational methods, and preliminary evidence suggests that such tools can yield reliable and valid indices of creativity (Acar & Runco, 2014;Dumas, Organisciak, & Doherty, 2020;Heinen & Johnson, 2018;Kenett, 2019;Prabhakaran, Green, & Gray, 2014).…”

“…Although it has limitations (Barbot, 2018), the AUT and other divergent thinking tests have shown consistent evidence of validity, with several studies reporting moderate to large correlations between AUT performance and real-world creative achievement in the arts and sciences (Beaty et al, 2018;Plucker, 1999). Paul Torrance, who developed a widely used creativity assessment (the Torrance Test of Creative Thinking; TTCT), provided perhaps the most compelling longitudinal evidence for the validity of divergent thinking tests: highly creative children-assessed by performance on the TTCT-grew up to be highly creative adults, reporting significantly more creative accomplishments when assessed decades later in adulthood (Plucker, 1999;Torrance, 1981); remarkably, a 50-year follow-up of Torrance's data further confirmed the validity of divergent thinking in predicting students' future creative accomplishment (Runco, Millar, Acar, & Cramond, 2010).…”

Creativity and Need for Cognition in Students of Pedagogy

“…"creativity" is a broad term which can be analyzed from the perspective of four paradigms. in its attributive dimension, we are dealing with "creativity" when the result of the act of creation is something new, useful and valuable in a given historical, cultural and political context (Kasof 1995;Runco & Jaeger , 2012;Sethy , 2009;acar & Runco , 2014;Barbot, 2018Barbot, , 2019tien, chang & Kuo, 2019;puryear, Kettler & Rinn, 2019; Education and Self development. Volume 14, № 3, 2019 creative commons by the authors is licenced under cc-BY-N c-N d Runco & Beghetto, 2019).…”