Model-free control for soft manipulators based on reinforcement learning

You, Xuanke; Zhang, Yixiao; Chen, Xiaotong; Liu, Xinghua; Wang, Zhanchi; Jiang, Hao; Chen, Xiaoping

doi:10.1109/iros.2017.8206123

Cited by 60 publications

(31 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Control policies obtained using RL techniques are also more robust to external disturbances making them ideal for the BR 2 manipulator, whose workspace is dependent on external loads. RL implementations in the context of soft robots is relatively new and has focused on traditional Q-learning [30], [31]. Both implementations work on controlling planar 2D motions, and have a relatively small state-action space and use fixed steps during transitions.…”

Section: Introductionmentioning

confidence: 99%

Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning

Satheeshbabu

Uppalapati

Chowdhary

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Soft robots undergo large nonlinear spatial deformations due to both inherent actuation and external loading. The physics underlying these deformations is complex, and often requires intricate analytical and numerical models. The complexity of these models may render traditional modelbased control difficult and unsuitable. Model-free methods offer an alternative for analyzing the behavior of such complex systems without the need for elaborate modeling techniques. In this paper, we present a model-free approach for open loop position control of a soft spatial continuum arm, based on deep reinforcement learning. The continuum arm is pneumatically actuated and attains a spatial workspace by a combination of unidirectional bending and bidirectional torsional deformation. We use Deep-Q Learning with experience replay to train the system in simulation. The efficacy and robustness of the control policy obtained from the system is validated both in simulation and on the continuum arm prototype for varying external loading conditions.

show abstract

Section: Introductionmentioning

confidence: 99%

Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning

Satheeshbabu

Uppalapati

Chowdhary

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…At present, the application of reinforcement learning in controlling soft arms is still in the early stage. In our previous work (You et al, 2017), control of the tip of the soft arm is implemented in a 2D plane based on Q -learning and its robustness to destruction of actuators is demonstrated. Satheeshbabu et al (2019) used a Deep Q -Network (DQN) method to implement open-loop positional control of the tip of a soft arm in 3D spaces.…”

Section: Related Workmentioning

confidence: 99%

“…One of the problems to use reinforcement learning to control soft-bodied arms is that the data is hard to obtain. For example, You et al (2017) collected data on a physical platform, which costs a lot. In order to expedite training, Satheeshbabu et al (2019) used a mathematical model presented in Uppalapati and Krishnan (2021) to generate virtual training data.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical control of soft manipulators towards unstructured interactions

Jiang

Wang

Jin

et al. 2021

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

Performing daily interaction tasks such as opening doors and pulling drawers in unstructured environments is a challenging problem for robots. The emergence of soft-bodied robots brings a new perspective to solving this problem. In this paper, inspired by humans performing interaction tasks through simple behaviors, we propose a hierarchical control system for soft arms, in which the low-level controller achieves motion control of the arm tip, the high-level controller controls the behaviors of the arm based on the low-level controller, and the top-level planner chooses what behaviors should be taken according to tasks. To realize the motion control of the soft arm in interacting with environments, we propose two control methods. The first is a feedback control method based on a simplified Jacobian model utilizing the motion laws of the soft arm that are not affected by environments during interaction. The second is a control method based on [Formula: see text]-learning, in which we present a novel method to increase training data by setting virtual goals. We implement the hierarchical control system on a platform with the Honeycomb Pneumatic Networks Arm (HPN Arm) and validate the effectiveness of this system on a series of typical daily interaction tasks, which demonstrates this proposed hierarchical control system could render the soft arms to perform interaction tasks as simply as humans, without force sensors or accurate models of the environments. This work provides a new direction for the application of soft-bodied arms and offers a new perspective for the physical interactions between robots and environments.

show abstract

“…ese trunks consist of multiple segments with dual actuation, i.e., electric motor and pneumatic [21][22][23][24][25][26][27][28]. In the current research scenario, the third-generation continuum robot, which is known as the bionic handling assistant (BHA) model developed by Festo [29][30][31], has entered the production environment. is is an advanced prototype that is constructed using the concepts of lightweight design and possesses the capability to operate with increased flexibility.…”

Section: Structurementioning

confidence: 99%

Continuum Robots for Manipulation Applications: A Survey

Kolachalama

Lakshmanan

2020

Journal of Robotics

View full text Add to dashboard Cite

This paper presents a literature survey documenting the evolution of continuum robots over the past two decades (1999–present). Attention is paid to bioinspired soft robots with respect to the following three design parameters: structure, materials, and actuation. Using this three-faced prism, we identify the uniqueness and novelty of robots that have hitherto not been publicly disclosed. The motivation for this study comes from the fact that continuum soft robots can make inroads in industrial manufacturing, and their adoption will be accelerated if their key advantages over counterparts with rigid links are clear. Four different taxonomies of continuum robots are included in this study, enabling researchers to quickly identify robots of relevance to their studies. The kinematics and dynamics of these robots are not covered, nor is their application in surgical manipulation.

show abstract

Model-free control for soft manipulators based on reinforcement learning

Cited by 60 publications

References 12 publications

Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning

Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning

Hierarchical control of soft manipulators towards unstructured interactions

Continuum Robots for Manipulation Applications: A Survey

Contact Info

Product

Resources

About