Exploring Co-creative Drawing Workflows

Jansen, Chipp; Sklar, Elizabeth

doi:10.3389/frobt.2021.577770

Cited by 9 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are witnessing human-as-collaborator applications in a wide range of art, ranging from a painting to a film. Collaborative drawing and painting workflows utilize AI and robotic arms to co-create with humans (Jansen and Sklar 2021). In AIBO, the opera performer's spoken words were used to generate texts from GPT-2 (Pearlman 2021).…”

Section: Human As Collaboratormentioning

confidence: 99%

Human in the Loop for Machine Creativity

Chung¹

2021

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) is increasingly utilized in synthesizing visuals, texts, and audio. These AI-based works, often derived from neural networks, are entering the mainstream market, as digital paintings, songs, books, and others. We conceptualize both existing and future human-in-the-loop (HITL) approaches for creative applications and to develop more expressive, nuanced, and multimodal models. Particularly, how can our expertise as curators and collaborators be encoded in AI models in an interactive manner? We examine and speculate on long term implications for models, interfaces, and machine creativity. Our selection, creation, and interpretation of AI art inherently contain our emotional responses, cultures, and contexts. Therefore, the proposed HITL may help algorithms to learn creative processes that are much harder to codify or quantify. We envision multimodal HITL processes, where texts, visuals, sounds, and other information are coupled together, with automated analysis of humans and environments. Overall, these HITL approaches will increase interaction between human and AI, and thus help the future AI systems to better understand our own creative and emotional processes.

show abstract

Section: Human As Collaboratormentioning

confidence: 99%

Human in the Loop for Machine Creativity

Chung¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While FRIDA aims at giving users control over content generation, the users are allowed to add their input only with an initial input prompt or image, after which they are excluded from the creative process. While it is still debatable whether such an autonomous creation is desired by humans practicing art [2], there is strong evidence of the potential value of a co-creative agent [3], [4], [5], [6], [7], [8], [9] specifically in the domain of art therapy [10], [11], [12], [13]. The benefits can be further increased when paired with a physical embodiment of such an agent and drawing in the real world [14], [15].…”

Section: Introductionmentioning

confidence: 99%

Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent

Schaldenbrand

2021

AAAI

View full text Add to dashboard Cite

The objective of most Reinforcement Learning painting agents is to minimize the loss between a target image and the paint canvas. Human painter artistry emphasizes important features of the target image rather than simply reproducing it. Using adversarial or L2 losses in the RL painting models, although its final output is generally a work of finesse, produces a stroke sequence that is vastly different from that which a human would produce since the model does not have knowledge about the abstract features in the target image. In order to increase the human-like planning of the model without the use of expensive human data, we introduce a new loss function for use with the model's reward function: Content Masked Loss. In the context of robot painting, Content Masked Loss employs an object detection model to extract features which are used to assign higher weight to regions of the canvas that a human would find important for recognizing content. The results, based on 332 human evaluators, show that the digital paintings produced by our Content Masked model show detectable subject matter earlier in the stroke sequence than existing methods without compromising on the quality of the final painting. Our code is available at https://github.com/pschaldenbrand/ContentMaskedLoss.

show abstract

Predicting Artist Drawing Activity via Multi-camera Inputs for Co-creative Drawing

Jansen

Sklar

2021

Towards Autonomous Robotic Systems

View full text Add to dashboard Cite

Exploring Co-creative Drawing Workflows

Cited by 9 publications

References 31 publications

Human in the Loop for Machine Creativity

Human in the Loop for Machine Creativity

Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent

Predicting Artist Drawing Activity via Multi-camera Inputs for Co-creative Drawing

Contact Info

Product

Resources

About