Integration of an actor-critic model and generative adversarial networks for a Chinese calligraphy robot

Wu, Ruiqi; Zhou, Changle; Chao, Fei; Yang, Longzhi; Lin, Chih Min; Shang, Changjing

doi:10.1016/j.neucom.2020.01.043

Cited by 21 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Robotik özelinde OUS incelendiğinde, RL'de bulunan etken (agent) yapısının keşfi için OUS kullanılmıştır [13]. Ayrıca, Çince kaligrafi robotu oluşturulması sırasında OUS bir gürültü terimi olarak kullanılmıştır [14].…”

Section: Ddpg Algoritmasında Bulunan Ornstein-uhlenbeckunclassified

DDPG Algoritmasında Bulunan Ornstein–Uhlenbeck Gürültüsünün Standart Sapmasının Araştırılması

Bingöl

2021

Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım Ve Teknoloji

View full text Add to dashboard Cite

Reinforcement learning is a learning method that many creatures often unwittingly use to gain abilities such as eating and walking. Inspired by this learning method, machine learning researchers have reduced this learning method to subheadings as value learning and policy learning. In this study, the noise standard deviation of the deep deterministic policy gradient (DDPG) method, which is one of the policy learning algorithms, was examined to solve inverse kinematics of a 2 degrees-of-freedom planar robot. For this examination, 8 different functions were determined depending on the maximum value of the output of the action artificial neural network. Created artificial neural networks were trained by using these functions in 1000 iterations with 200 steps in each iteration. After the training, the statistical difference between the groups was examined and it was found that there was no statistical difference between the three best groups. For this reason, the best three groups were retrained 2500 iterations and 200 steps and tested for 100 different test scenarios after the training. After testing, the inverse kinematic equation of the 2 degrees-of-freedom planar robot with minimal errors was obtained with the help of artificial neural networks. In the light of the results, the importance of the choice of the standard deviation of noise and the correct range of selection was presented for researchers who will work in this field.

show abstract

Section: Ddpg Algoritmasında Bulunan Ornstein-uhlenbeckunclassified

DDPG Algoritmasında Bulunan Ornstein–Uhlenbeck Gürültüsünün Standart Sapmasının Araştırılması

Bingöl

2021

Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım Ve Teknoloji

View full text Add to dashboard Cite

show abstract

“…Chao et al [20] proposed an arm robot that can draw Chinese calligraphy. Recently, Wu et al [21] have proposed its extended version. Their models are based on GAN and thus very similar to GAIL; however, their purpose is rather to realize a practical robotic system and not focused on the analysis of the learned rewards to understand the trends and characteristics of handwriting trajectories.…”

Section: B Reinforcement Learning For Handwritingmentioning

confidence: 99%

What is the Reward for Handwriting? -- Handwriting Generation by Imitation Learning

Kanda,

Iwana,

Uchida

2020

Preprint

View full text Add to dashboard Cite

Analyzing the handwriting generation process is an important issue and has been tackled by various generation models, such as kinematics based models and stochastic models. In this study, we use a reinforcement learning (RL) framework to realize handwriting generation with the careful future planning ability. In fact, the handwriting process of human beings is also supported by their future planning ability; for example, the ability is necessary to generate a closed trajectory like '0' because any shortsighted model, such as a Markovian model, cannot generate it. For the algorithm, we employ generative adversarial imitation learning (GAIL). Typical RL algorithms require the manual definition of the reward function, which is very crucial to control the generation process. In contrast, GAIL trains the reward function along with the other modules of the framework. In other words, through GAIL, we can understand the reward of the handwriting generation process from handwriting examples. Our experimental results qualitatively and quantitatively show that the learned reward catches the trends in handwriting generation and thus GAIL is well suited for the acquisition of handwriting behavior.

show abstract

“…Most of the existing robotic writing systems do not have a realistic aesthetic evaluation mechanism to critically assess the writing performance [3]. Instead, several studies adopted the Fréchet inception distances (FID) or the restoring accuracy of an Autoencoder network to represent the aesthetic performance [4], [26], [27]. These methods well examined the writing results based on distribution, but often ignored the structural information of letters or numerals representing diversity.…”

Section: Introductionmentioning

confidence: 99%

Solving Robotic Trajectory Sequential Writing Problem via Learning Character’s Structural and Sequential Information

Guo

Chao

et al. 2024

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

The writing sequence of numerals or letters often affects aesthetic aspects of the writing outcomes. As such, it remains a challenge for robotic calligraphy systems to perform, mimicking human writers' implicit intention. This paper presents a new robot calligraphy system that is able to learn writing sequences with limited sequential information, producing writing results compatible to human writers with good diversity. In particular, the system innovatively applies a Gated Recurrent Unit (GRU) network to generate robotic writing actions with the support of a pre-labeled trajectory sequence vector. Also, a new evaluation method is proposed that considers the shape, trajectory sequence, and structural information of the writing outcome, thereby helping ensure the writing quality. A swarm optimization algorithm is exploited to create an optimal set of parameters of the proposed system.The proposed approach is evaluated using Arabic numerals, and the experimental results demonstrate the competitive writing performance of the system against state-of-the-art approaches regarding multiple criteria (including FID, MAE, PSNR, SSIM, and PerLoss), as well as diversity performance concerning variance and entropy. Importantly, the proposed GRU-based robotic motion planning system, supported with swarm optimization can learn from a small dataset, while producing calligraphy writing with diverse and aesthetically pleasing outcomes.

show abstract

Integration of an actor-critic model and generative adversarial networks for a Chinese calligraphy robot

Cited by 21 publications

References 35 publications

DDPG Algoritmasında Bulunan Ornstein–Uhlenbeck Gürültüsünün Standart Sapmasının Araştırılması

DDPG Algoritmasında Bulunan Ornstein–Uhlenbeck Gürültüsünün Standart Sapmasının Araştırılması

What is the Reward for Handwriting? -- Handwriting Generation by Imitation Learning

Solving Robotic Trajectory Sequential Writing Problem via Learning Character’s Structural and Sequential Information

Contact Info

Product

Resources

About