Measuring Creativity in Academic Writing: An Analysis of Essays in Advanced Placement Language and Composition

Abstract

The demand for a creative workforce has never been higher, yet schools struggle to teach and assess creativity among students efficiently. Compositions are an effective way to incorporate creativity across the curriculum; however, essays are time consuming to evaluate for quality or creativity. This study explored (a) if high creativity scores are related to high quality and sophistication in academic writing, and (b) if extant text-mining tools effectively identify quality, sophistication, and creativity in academic essays. Four teacher raters analyzed quality, sophistication, and creativity of 230 essays written by students aged 15–17 for Advanced Placement Language and Composition. We also used text-mining tools (e.g., semantic distance, Shannon's entropy, idea density) to score these essays. Teacher-rated creativity scores correlated with quality and sophistication scores, as well as with some of the text-mining tools, suggesting that these tools can capture quality and sophistication in addition to creativity. Implications for educational practice are discussed.

Testing the Domain Specificity of Creativity with Kaufman Domains of Creativity Scale: A Meta‐Analytic Confirmatory Factor Analysis

“…One hurdle for this type of work is the limited number of domains typically assessed using expert raters; there are typically only two or three assessed in the same study and are usually largely confined to writing and art (Cseh & Jeffries, 2019;Kaufman & Baer, 2012). However, with advances in automated scoring of tasks beyond divergent thinking tasks (e.g., see Bower, Acar, & Kursuncu, 2023;Johnson et al, 2022 for essays or stories), future research may be better able to address these issues. Certainly, when it is possible to consider multiple sources of assessment, a more comprehensive and accurate understanding of the interplay between different domains of creativity can be achieved.…”