Robot Calligraphy using Pseudospectral Optimal Control in Conjunction with a Novel Dynamic Brush Model

Wang, Sen; Chen, Jiaqi; Deng, Xuanliang; Hutchinson, Seth; Dellaert, Frank

doi:10.1109/iros45743.2020.9341787

Cited by 21 publications

(12 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wang et al . [WCD*20] proposed to simulate calligraphy by collecting data with a robotic arm. Bidgoli et al .…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Data‐Driven Ink Painting Brushstroke Rendering

Madono,

Simo‐Serra

2023

Computer Graphics Forum

View full text Add to dashboard Cite

Although digital painting has advanced much in recent years, there is still a significant divide between physically drawn paintings and purely digitally drawn paintings. These differences arise due to the physical interactions between the brush, ink, and paper, which are hard to emulate in the digital domain. Most ink painting approaches have focused on either using heuristics or physical simulation to attempt to bridge the gap between digital and analog, however, these approaches are still unable to capture the diversity of painting effects, such as ink fading or blotting, found in the real world. In this work, we propose a data‐driven approach to generate ink paintings based on a semi‐automatically collected high‐quality real‐world ink painting dataset. We use a multi‐camera robot‐based setup to automatically create a diversity of ink paintings, which allows for capturing the entire process in high resolution, including capturing detailed brush motions and drawing results. To ensure high‐quality capture of the painting process, we calibrate the setup and perform occlusion‐aware blending to capture all the strokes in high resolution in a robust and efficient way. Using our new dataset, we propose a recursive deep learning‐based model to reproduce the ink paintings stroke by stroke while capturing complex ink painting effects such as bleeding and mixing. Our results corroborate the fidelity of the proposed approach to real hand‐drawn ink paintings in comparison with existing approaches. We hope the availability of our dataset will encourage new research on digital realistic ink painting techniques.

show abstract

“…Wang et al . [WCD*20] proposed to simulate calligraphy by collecting data with a robotic arm. Bidgoli et al .…”

Section: Related Workmentioning

confidence: 99%

“…Wang et al . [WCD*20] only capture the final result of calligraphy whereas Bidgoli et al . [BDGH*20] only captured single brush strokes.…”

Section: Related Workmentioning

confidence: 99%

Data‐Driven Ink Painting Brushstroke Rendering

Madono,

Simo‐Serra

2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Currently, three methods are widely applied for Chinese character extraction, i.e., (a) computer font reproduction, (b) imitation of human writing trajectories and (c) Chinese character image decomposition. Computer font reproduction involves using the stroke information that comes with a standard font library [5][6][7][8] and handling of word posters [9][10][11][12][13], but these methods are database dependent and cannot imitate the individual writing of a particular person. Imitation of human writing trajectories includes recording the trajectory of the pen tip when a person is writing [14,15].…”

Section: Introductionmentioning

confidence: 99%

Image-based anti-interference robotic Chinese character writing system

Li,

Yang,

et al. 2024

Robotica

View full text Add to dashboard Cite

This article designs a robotic Chinese character writing system that can resist random human interference. Firstly, an innovative stroke extraction method of Chinese characters was devised. A basic Chinese character stroke extraction method based on cumulative direction vectors is used to extract the components that make up the strokes of Chinese characters. The components are then stitched together into strokes based on the sequential base stroke joining method. To enable the robot to imitate handwriting Chinese character skills, we utilised stroke information as the demonstration and modelled the skills using dynamic movement primitives (DMPs). To suppress random human interference, this article combines improved DMPs and conductance control to adjust robot trajectories based on real-time visual measurements. The experimental results show that the proposed method can accurately extract the strokes of most Chinese characters. The designed trajectory adjustment method offers better smoothness and robustness than direct rotating and translating curves. The robot is able to adjust its posture and trajectory in real time to eliminate the negative impacts of human interference.

show abstract

“…The most common approach is to use painterly rendering algorithms such as [1], [2] to generate brush stroke placements which are then executed by a serial manipulator [3]. Some more recent approaches build on the traditional painterly rendering approach with learned brush models [3], optimization [4], differentiable rendering [5], [6], and semantic objective functions [7], [8] to generate higher quality renderings. Other works apply generative models to generate brush strokes directly [9], [10].…”

Section: Introductionmentioning

confidence: 99%

GTGraffiti: Spray Painting Graffiti Art from Human Painting Motions with a Cable Driven Parallel Robot

Chen

Baek

Florez

et al. 2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Robot art presents an opportunity to both showcase and advance state-of-the-art robotics through the challenging task of creating art. Creating large-scale artworks in particular engages the public in a way that small-scale works cannot, and the distinct qualities of brush strokes contribute to an organic and human-like quality. Combining the large scale of murals with the strokes of the brush medium presents an especially impactful result, but also introduces unique challenges in maintaining precise, dextrous motion control of the brush across such a large workspace. In this work, we present the first robot to our knowledge that can paint architectural-scale murals with a brush. We create a hybrid robot consisting of a cable-driven parallel robot and 4 degree of freedom (DoF) serial manipulator to paint a 27m by 3.7m mural on windows spanning 2-stories of a building. We discuss our approach to achieving both the scale and accuracy required for brush-painting a mural through a combination of novel mechanical design elements, coordinated planning and control, and on-site calibration algorithms with experimental validations.

show abstract

Robot Calligraphy using Pseudospectral Optimal Control in Conjunction with a Novel Dynamic Brush Model

Cited by 21 publications

References 33 publications

Data‐Driven Ink Painting Brushstroke Rendering

Data‐Driven Ink Painting Brushstroke Rendering

Image-based anti-interference robotic Chinese character writing system

GTGraffiti: Spray Painting Graffiti Art from Human Painting Motions with a Cable Driven Parallel Robot

Contact Info

Product

Resources

About