Learning to Paint With Model-based Deep Reinforcement Learning

Huang, Zhewei; Wen, Hui; Zhou, Shuchang

doi:10.48550/arxiv.1903.04411

Cited by 17 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We first explore Generative Adversarial Networks (GANs) [7] which have been successfully used as generative models for synthesizing high quality images [15,16,29], and has also seen creative applications such as image-to-image translation [30], controllable Anime character generation [8,13,14], photo-realistic face Figure 8: Painting sequences produced by learning to paint [12], a neural painting model, on a few exemplary images in the KaoKore dataset. On each row, the leftmost image is the reference image from theKaoKore dataset, while the smaller images illustrate the generated painting sequence.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…To give the synthesis process a more artwork-like inductive bias, we consider Stroke-based rendering [10] which produces a reference painting by sequentially drawing primitives, such as simple strokes, onto a canvas. Recent advances using neural networks have been proposed by integrating techniques such as differentiable image parameterizations [21,22] or reinforcement learning [6,12], which can greatly improve the quality Figure 9: Final canvases produced by learning to paint [12] (second row) after all painting steps have been completed in order to approximate the reference image (first row). Reference images match Figure 6 for easier comparison of style studies.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…Figure 10: More examples of learning to paint [12] on KaoKore images after all painting steps. Reference images match Figure 7 for easier comparison of style studies.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…Given that our proposed KaoKore dataset is based on art collections, we are interested in applying these neural painting methods with image production mechanisms that better resemble human artists. In particular, we explore applying intrinsic style transfer [22] and learning to paint [12] on the proposed KaoKore dataset. [22] is a set of methods that combines differentiable approximation with non-differentiable painting primitives (e.g., strokes in our colored pencil drawing style) and an adversarially trained neural agent that learns to recreate the reference image on the canvas by producing these primitives sequentially.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…Learning to paint [12] is a neural painting model which differentiates itself from others in a few aspects, including using regions marked by quadratic Bézier curves as painting primitives as well as leveraging model-based deep reinforcement learning for training. As a result, its painting sequence is made of simple regions rather than brush-like strokes, and the sequence is as short as possible due to the optimization goal used in reinforcement learning training.…”

Section: Intrinsic Style Transfermentioning

confidence: 99%

See 4 more Smart Citations

KaoKore: A Pre-modern Japanese Art Facial Expression Dataset

Tian¹,

Suzuki²,

Clanuwat³

et al. 2020

Preprint

View full text Add to dashboard Cite

From classifying handwritten digits to generating strings of text, the datasets which have received long-time focus from the machine learning community vary greatly in their subject matter. This has motivated a renewed interest in building datasets which are socially and culturally relevant, so that algorithmic research may have a more direct and immediate impact on society. One such area is in history and the humanities, where better and relevant machine learning models can accelerate research across various fields. To this end, newly released benchmarks [3] and models [4] have been proposed for transcribing historical Japanese cursive writing, yet for the field as a whole using machine learning for historical Japanese artworks still remains largely uncharted. To bridge this gap, in this work we propose a new dataset KaoKore 1 which consists of faces extracted from pre-modern Japanese artwork. We demonstrate its value as both a dataset for image classification as well as a creative and artistic dataset, which we explore using generative models.

show abstract

Section: Generative Adversarial Networkmentioning

confidence: 99%

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…Figure 10: More examples of learning to paint [12] on KaoKore images after all painting steps. Reference images match Figure 7 for easier comparison of style studies.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

Section: Generative Adversarial Networkmentioning

confidence: 99%

Section: Intrinsic Style Transfermentioning

confidence: 99%

See 3 more Smart Citations

KaoKore: A Pre-modern Japanese Art Facial Expression Dataset

Tian¹,

Suzuki²,

Clanuwat³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Synthetic Data for Deep Learning

Nikolenko

2021

Springer Optimization and Its Applications

220

View full text Add to dashboard Cite

Aims and ScopeOptimization has continued to expand in all directions at an astonishing rate. New algorithmic and theoretical techniques are continually developing and the diffusion into other disciplines is proceeding at a rapid pace, with a spot light on machine learning, artificial intelligence, and quantum computing. Our knowledge of all aspects of the field has grown even more profound. At the same time, one of the most striking trends in optimization is the constantly increasing emphasis on the interdisciplinary nature of the field. Optimization has been a basic tool in areas not limited to applied mathematics, engineering, medicine, economics, computer science, operations research, and other sciences.The series Springer Optimization and Its Applications (SOIA) aims to publish state-of-the-art expository works (monographs, contributed volumes, textbooks, handbooks) that focus on theory, methods, and applications of optimization. Topics covered include, but are not limited to, nonlinear optimization, combinatorial optimization, continuous optimization, stochastic optimization, Bayesian optimization, optimal control, discrete optimization, multi-objective optimization, and more. New to the series portfolio include Works at the intersection of optimization and machine learning, artificial intelligence, and quantum computing.Volumes from this series are indexed by Web of Science, zbMATH, Mathematical Reviews, and SCOPUS.

show abstract