Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence

Abstract

While developing a story, novices and published writers alike have had to look outside themselves for inspiration. Language models have recently been able to generate text fluently, producing new stochastic narratives upon request. However, effectively integrating such capabilities with human cognitive faculties and creative processes remains challenging. We propose to investigate this integration with a multimodal writing support interface that offers writing suggestions textually, visually, and aurally. We conduct an extensive study that combines elicitation of prior expectations before writing, observation and semi-structured interviews during writing, and outcome evaluations after writing. Our results illustrate individual and situational variation in machine-in-the-loop writing approaches, suggestion acceptance, and ways the system is helpful. Centrally, we report how participants perform integrative leaps , by which they do cognitive work to integrate suggestions of varying semantic relevance into their developing stories. We interpret these findings, offering modeling and design recommendations for future creative writing support technologies.

Sparks: Inspiration for Science Writing using Language Models

“…More recent work has varied the ways in which technology can support the writer, for instance by providing description, plot points, or even asking questions, depending on the desires of the writer [3,15]. Singh et al [52] investigates a multimodal system for story writing, that includes language model suggestions, and discusses the integrative leaps writers make when incorporating suggestions. Lee et al [33] presents a large dataset of how writers incorporate suggestions from GPT-3 in response to creative writing and argumentative writing prompts, quantifying measures like how many times suggestions with named entities were incorporated and how mutual turntaking was during the writing process.…”

Examining Agency, Expertise, and Roles of AI Systems in AI-Mediated Communication

“…However, the AI system may have been incorporated into the person's sense of agency because it was under their control and assisted them in goal fulfillment [78]. For instance, a study by Singh et al [98] noted that some participants' assessments of agency revolved around deciding to use the system at all, not how much AI-generated text they accepted into their messages or the content of the AI-generated text. This suggests a perception of the AI system as an extension of the self; agency was not impacted by how much the person collaborated with the system, as it would be if they were collaborating with another person.…”

Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision

“…Another line of work by Sun et al (2021); Singh et al (2022) on creative writing designed humanmachine interaction interfaces to encourage new content generation. However, text revision focuses on improving the quality of existing writing and keeping the original content as much as possible.…”