Artistic Style in Robotic Painting; a Machine Learning Approach to Learning Brushstroke from Human Artists

Kim

2022 International Conference on Robotics and Automation (ICRA)

et al. 2022

We present an automated learning framework for a robotic sketching agent that is capable of learning stroke-based rendering and motor control simultaneously. We formulate the robotic sketching problem as a deep decoupled hierarchical reinforcement learning; two policies for strokebased rendering and motor control are learned independently to achieve sub-tasks for drawing, and form a hierarchy when cooperating for real-world drawing. Without hand-crafted features, drawing sequences or trajectories, and inverse kinematics, the proposed method trains the robotic sketching agent from scratch. We performed experiments with a 6-DoF robot arm with 2F gripper to sketch doodles. Our experimental results show that the two policies successfully learned the sub-tasks and collaborated to sketch the target images. Also, the robustness and flexibility were examined by varying drawing tools and surfaces.

Section: B Robotic Drawingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent

Kim

2022 International Conference on Robotics and Automation (ICRA)

et al. 2022

“…The capabilities of the painting robot have been extended to the palette knife technique in [23]. Further examples of recent robotic systems for artistic purposes are given by Skywork-daVinci, a painting support based on a human-in-the-loop mechanism [24], the robotic painting setup capable of learning brushstrokes from human artists presented in [25], and the interactive multi-robot painting system described in [26].…”

Section: Related Workmentioning

confidence: 99%

Human–Robot Interaction through Eye Tracking for Artistic Drawing

et al. 2021

In this paper, authors present a novel architecture for controlling an industrial robot via an eye tracking interface for artistic purposes. Humans and robots interact thanks to an acquisition system based on an eye tracker device that allows the user to control the motion of a robotic manipulator with his gaze. The feasibility of the robotic system is evaluated with experimental tests in which the robot is teleoperated to draw artistic images. The tool can be used by artists to investigate novel forms of art and by amputees or people with movement disorders or muscular paralysis, as an assistive technology for artistic drawing and painting, since, in these cases, eye motion is usually preserved.

Proceedings of the Institution of Mechanical Engineers, Part C:

“…To enable the robot to paint similar to an artist's style, a machine learning approach was developed by Bidgoli to learn brushstrokes from human artists by collecting samples of the artist's strokes for training. 9 In addition, to make the painting look closer to the artwork of humans Karimov et al considered aerial perspective, gamut compression, and brushstroke coherence in image preprocessing. 10 Many cases of painting with robotic arms have occurred in recent years; however, most of them have been used for oil painting, 2 watercolour painting, 11 Chinese painting, 12 etc.,.…”

Section: Introductionmentioning

confidence: 99%

Artistic robotic pencil sketching using closed-loop force control

Hung

Shaw

2022

This study introduces an artistic robotic painting system. The main purpose was to use a force sensor combined with a 6-axis robotic arm to create pencil sketches. Compared with other painting robots, our system has additional tactile sensing that can overcome the problem of pencil wear and can simulate the feeling of human sketches. To provide the robot with the capability to create paintings, our experiment used neural style transfer technology to extract the content and style image features and recomposed them into a new artwork. The newly generated image was processed for edge detection and further layered using methods, such as k-means clustering and image grayscale quantisation, resulting in a contour layer and multiple background layers. We carried out path planning of different layers according to the layering result, and we sorted the pixels by finding the shortest distance between them to plan a feasible path for the robotic arm at a higher speed. Finally, in the robotic arm painting experiment, we conducted two types of art creation: Chinese brush painting and pencil sketching. In the first type, we used position control to create a Chinese brush painting, and in the second type, we used a proportional-integral-derivative (PID) controller for force control to create pencil sketching. The experimental results show the validity of the proposed techniques in artistic robotic painting, both in Chinese brush painting and pencil sketching, with painting times of less than 30 min.