Model-based reinforcement learning approach for deformable linear object manipulation

Han, Haifeng; Paul, Gavin; Matsubara, Takamitsu

doi:10.1109/coase.2017.8256194

Cited by 31 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed action-selection algorithm for this work is rooted in model-based reinforcement learning. Reinforcement learning (RL) has been used to manipulate linear and planar objects using both model-free [27], [11], [28] and model-based approaches [29], [30]. Much of this prior work relies on simulation in order to collect training data, as physical experiments are expensive.…”

Section: Model-based Reinforcement Learningmentioning

confidence: 99%

Deformable Elasto-Plastic Object Shaping using an Elastic Hand and Model-Based Reinforcement Learning

Matl¹,

Bajcsy²

2021

Preprint

View full text Add to dashboard Cite

Deformable solid objects such as clay or dough are prevalent in industrial and home environments. However, robotic manipulation of such objects has largely remained unexplored in literature due to the high complexity involved in representing and modeling their deformation. This work addresses the problem of shaping elasto-plastic dough by proposing to use a novel elastic end-effector to roll dough in a reinforcement learning framework. The transition model for the end-effector-to-dough interactions is learned from one hour of robot exploration, and doughs of different hydration levels are rolled out into varying lengths. Experimental results are encouraging, with the proposed framework accomplishing the task of rolling out dough into a specified length with 60% fewer actions than a heuristic method. Furthermore, we show that estimating stiffness using the soft end-effector can be used to effectively initialize models, improving robot performance by approximately 40% over incorrect model initialization.

show abstract

Section: Model-based Reinforcement Learningmentioning

confidence: 99%

Deformable Elasto-Plastic Object Shaping using an Elastic Hand and Model-Based Reinforcement Learning

Matl¹,

Bajcsy²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In follow-up works, Ebert et al [10,11] investigate different image losses. Han et al [12] uses model-based reinforcement learning (RL) for deforming a rope in 2D with fixed start and goal configurations. While these works have demonstrated short-horizon deformation tasks, the method does not directly extend to knot planning, where the goal state is abstract, represented by topology.…”

Section: Related Workmentioning

confidence: 99%

Learning Topological Motion Primitives for Knot Planning

Yan

Zhu

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, we approach the challenging problem of motion planning for knot tying. We propose a hierarchical approach in which the top layer produces a topological plan and the bottom layer translates this plan into continuous robot motion. The top layer decomposes a knotting task into sequences of abstract topological actions based on knot theory. The bottom layer translates each of these abstract actions into robot motion trajectories through learned topological motion primitives. To adapt each topological action to the specific rope geometry, the motion primitives take the observed rope configuration as input. We train the motion primitives by imitating human demonstrations and reinforcement learning in simulation. To generalize human demonstrations of simple knots into more complex knots, we observe similarities in the motion strategies of different topological actions and design the neural network structure to exploit such similarities. We demonstrate that our learned motion primitives can be used to efficiently generate motion plans for tying the overhand knot. The motion plan can then be executed on a real robot using visual tracking and Model Predictive Control. We also demonstrate that our learned motion primitives can be composed to tie a more complex pentagram-like knot despite being only trained on human demonstrations of simpler knots.

show abstract

“…Data-driven approaches have also been applied to approximate the deformation, without studying the complex dynamics of DLOs beforehand. A model-based reinforcement learning (RL) approach was proposed for robots to control the shape of the DLO in [17], with the current shape as the input and the manipulation policy as the output. In [18], a deep-neural-network-based dynamics model was trained to predict the future shape of the DLO given the current shape and the action.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Control for Robotic Manipulation of Deformable Linear Objects with Offline and Online Learning of Unknown Models

Yu¹,

Zhong²,

Zhong³

et al. 2021

Preprint

View full text Add to dashboard Cite

The deformable linear objects (DLOs) are common in both industrial and domestic applications, such as wires, cables, ropes. Because of its highly deformable nature, it is difficult for the robot to reproduce human's dexterous skills on DLOs. In this paper, the unknown deformation model is estimated in both the offline and online manners. The offline learning aims to provide a good approximation prior to the manipulation task, while the online learning aims to compensate the errors due to insufficient training (e.g. limited datasets) in the offline phase. The offline module works by constructing a series of supervised neural networks (NNs), then the online module receives the learning results directly and further updates them with the technique of adaptive NNs. A new adaptive controller is also proposed to allow the robot to perform manipulation tasks concurrently in the online phase. The stability of the closed-loop system and the convergence of task errors are rigorously proved with Lyapunov method. Simulation studies are presented to illustrate the performance of the proposed method.

show abstract

Model-based reinforcement learning approach for deformable linear object manipulation

Cited by 31 publications

References 15 publications

Deformable Elasto-Plastic Object Shaping using an Elastic Hand and Model-Based Reinforcement Learning

Deformable Elasto-Plastic Object Shaping using an Elastic Hand and Model-Based Reinforcement Learning

Learning Topological Motion Primitives for Knot Planning

Adaptive Control for Robotic Manipulation of Deformable Linear Objects with Offline and Online Learning of Unknown Models

Contact Info

Product

Resources

About