Learning to Paint With Model-Based Deep Reinforcement Learning

Huang, Zhewei; Zhou, Shuchang; Wen, Hui

doi:10.1109/iccv.2019.00880

Cited by 133 publications

(173 citation statements)

References 19 publications

Supporting

Mentioning

168

Contrasting

Unclassified

Order By: Relevance

“…10 RL could be useful in creative applications, where there may not be a predefined way to perform a given task, but where there are rules that the model has to follow to perform its duties correctly. Current applications involve end-to-end RL combined with CNNs, including gaming (Mnih et al 2013), and RLs with GANs in optimal painting stroke in stroke-based rendering (Huang et al 2019). Recently RL methods have been developed using a graph neural network (GNN) to play Diplomacy, a highly complex 7-player (large scale) board game (Anthony et al 2020).…”

Section: Deep Reinforcement Learning (Drl)mentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

This paper reviews the current state of the art in artificial intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically machine learning (ML) algorithms, is provided including convolutional neural networks (CNNs), generative adversarial networks (GANs), recurrent neural networks (RNNs) and deep Reinforcement Learning (DRL). We categorize creative applications into five groups, related to how AI technologies are used: (i) content creation, (ii) information analysis, (iii) content enhancement and post production workflows, (iv) information extraction and enhancement, and (v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, ML-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of ML in domains with fewer constraints, where AI is the ‘creator’, remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human-centric—where it is designed to augment, rather than replace, human creativity.

show abstract

Section: Deep Reinforcement Learning (Drl)mentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

show abstract

“…Nowadays artificial intelligent painting that an agent can paint strokes on a canvas in sequence to generate a painting that resembles the given target image. Some work has studied in teaching machines to learn painting-related skills, such as sketch, doodle and write characters (Huang 2019). Human-machine art has triggered our thinking about AI, showing the current state of human life through machines, and paying attention to the real living environment of people.…”

Section: Introductionmentioning

confidence: 99%

Research on human–AI co-creation based on reflective design practice

Zhou

2020

CCF Trans. Pervasive Comp. Interact.

View full text Add to dashboard Cite

Artificial intelligence has appeared in every aspect of our life, whether it is a large number of data analysis and repetitive work, or the scanning recognition in life, AI has become an essential application technology. How humans and AI will coexist in the future is a thought-provoking question. The involvement of AI will lead to changes in design method and process, the designers and researchers need to conduct research on future designs from a new perspective. We selected the field of design practice and put forward the concept of human-AI co-creation (HACC), aiming to study how AI can better design and create with people. We use ethnographic design research methods to summarize how teachers guide students in design practice, and summarize the normative vocabulary that AI guides students to use in experiments. Finally, we summarize the basic elements of HACC, and then use the "Wizard of OZ" method to simulate HACC experiment, and further improve the HACC model through video observation and data analysis.

show abstract

“…Deep reinforcement learning is a reinforcement learning framework based on deep learning, which was successfully applied to many computer vision applications [ 57 , 58 , 59 , 60 ]. Han et al [ 61 ] first attempted to apply enhanced cropping agent learning to determine the video object segmentation scheme.…”

Section: Related Workmentioning

confidence: 99%

ASNet: Auto-Augmented Siamese Neural Network for Action Recognition

Zhang

Xiong

et al. 2021

Sensors

View full text Add to dashboard Cite

Human action recognition methods in videos based on deep convolutional neural networks usually use random cropping or its variants for data augmentation. However, this traditional data augmentation approach may generate many non-informative samples (video patches covering only a small part of the foreground or only the background) that are not related to a specific action. These samples can be regarded as noisy samples with incorrect labels, which reduces the overall action recognition performance. In this paper, we attempt to mitigate the impact of noisy samples by proposing an Auto-augmented Siamese Neural Network (ASNet). In this framework, we propose backpropagating salient patches and randomly cropped samples in the same iteration to perform gradient compensation to alleviate the adverse gradient effects of non-informative samples. Salient patches refer to the samples containing critical information for human action recognition. The generation of salient patches is formulated as a Markov decision process, and a reinforcement learning agent called SPA (Salient Patch Agent) is introduced to extract patches in a weakly supervised manner without extra labels. Extensive experiments were conducted on two well-known datasets UCF-101 and HMDB-51 to verify the effectiveness of the proposed SPA and ASNet.

show abstract

Learning to Paint With Model-Based Deep Reinforcement Learning

Cited by 133 publications

References 19 publications

Artificial intelligence in the creative industries: a review

Artificial intelligence in the creative industries: a review

Research on human–AI co-creation based on reflective design practice

ASNet: Auto-Augmented Siamese Neural Network for Action Recognition

Contact Info

Product

Resources

About