A Robotic Writing Framework–Learning Human Aesthetic Preferences via Human–Machine Interactions

Abstract

Intelligent robots are required to fully understand human intentions and operations in order to support or collaborate with humans to complete complicated tasks, which is typically implemented by employing human-machine interaction techniques. This paper proposes a new robotic learning framework to perform numeral writing tasks by investigating human-machine interactions with human preferences. In particular, the framework implements a trajectory generative module using a generative adversarial network (GAN)-based method and develops a human preference feedback system to enable the robot to learn human preferences. In addition, a convolutional neural network, acting as a discriminative network, classifies numeral images to support the development of the basic numeral writing ability, and another convolutional neural network, acting as a human preference network, learns a human user's aesthetic preference by taking the feedback on two written numerical images during the training process. The experimental results show that the written numerals based on the preferences of ten users were different from those of the training data set and that the writing models with the preferences from different users generate numerals in different styles, as evidenced by the Fréchet inception distance (FID) scores. The FID scores of the proposed framework with a preference network were noticeably greater than those of the framework without a preference network. This phenomenon indicates that the human-machine interactions effectively guided the robotic system to learn different writing styles. These results prove that the proposed approach is able to enable the calligraphy robot to successfully write numerals in accordance with the preferences of a human user.INDEX TERMS Human-machine interaction, human preference, neural networks, robotic calligraphy, robotic writing trajectory.

Solving Robotic Trajectory Sequential Writing Problem via Learning Character’s Structural and Sequential Information

“…The loop number k of the GRU-based trajectory generator is set according to the writing complexity of each numeral, and the range of k is set from 3 to 8. Among them, the k values of numerals 0 to 9 are respectively set to: [7,3,5,7,4,8,7,3,7,7].The parameters in the paper are the best results obtained by tuning. The corresponding tuning process is not shown due to the length of the article.…”

“…Another key challenge in robotic calligraphy is the aesthetic evaluation of robotic writing results with particular considerations of diversity. Most of the existing robotic writing systems do not have a realistic aesthetic evaluation mechanism to critically assess the writing performance [3]. Instead, several studies adopted the Fréchet inception distances (FID) or the restoring accuracy of an Autoencoder network to represent the aesthetic performance [4], [26], [27].…”

A Chinese Calligraphy-Writing Robotic System Based on Image-to-Action Translations and a Hypothesis Generation Net

“…(7). Subscript (2 ) v p p  denotes the approximation value rounded to a positive integer. We utilized the Eq.…”

Automatic Dialing Test System Based on Visual Servo Control

“…Intelligent robot control has become more common in recent years, and the technologies include image processing, path planning, and arm control. (1)(2)(3)(4)(5)(6) In previous works, (7) mobile phone tests were performed using a fixed camera, so that mistakes had to be canceled and repointed. The main purpose of this study is to increase the accuracy of clicking buttons by means of visual servo control.…”