Control of Single-Segment Continuum Robots: Reinforcement Learning vs. Neural Network based PID

Chattopadhyay, Sudipta; Bhattacherjee, Saptak; Bandyopadhyay, Soutrik; Sengupta, Anindita; Bhaumik, Subhasis

doi:10.1109/iccpcct.2018.8574225

Cited by 6 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, data-driven control methods are considered useful for soft robots because of modeling difficulty, and the application of reinforcement learning has been proposed ( Bhagat et al, 2019 ). Data-driven methods are often based on machine learning and include sampling data by actually moving the robot and modeling it using machine learning ( Bruder et al, 2019 ; Buchler et al, 2018 ; George Thuruthel et al, 2017 ; Giorelli et al, 2015 ; Lee et al, 2017 ; Rolf and Steil, 2014 ; Thuruthel et al, 2017 ); and learning directly implemented on the controller by moving the robot and using reinforcement learning ( Chattopadhyay et al, 2018 ; Morimoto et al, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

“…There are many studies on reaching tasks using model-free reinforcement learning algorithms. They range from using continuum robot arms with one segment ( Chattopadhyay et al, 2018 ; Satheeshbabu et al, 2019 , 2020 ) to two ( Yang et al, 2019 ), three ( Zhang et al, 2017 ), and four ( You et al, 2017 ) segments. There are also other studies that use multi-agent reinforcement learning in which each actuator of a multi-degree-of-freedom arm is considered as one agent ( Ansari et al, 2018 ; Perrusquía et al, 2020 ; Ji et al, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

“…Accordingly, learning using reinforcement learning can potentially solve the problem of soft robot control by developing data-driven control methods ( Bhagat et al, 2019 ). Several studies have been conducted on reaching tasks using reinforcement learning ( You et al, 2017 ; Zhang et al, 2017 ; Chattopadhyay et al, 2018 ; Satheeshbabu et al, 2019 ; Fetchrobotics, 2020 ; Morimoto et al, 2021 ). However, the research on soft robots and continuum robot arms, especially with regard to reinforcement learning, depends on the continuum robot arms owned by each research group, and there is no investigation into comparison with conventional “rigid” robots.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Characterization of continuum robot arms under reinforcement learning and derived improvements

et al. 2022

View full text Add to dashboard Cite

In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods.

show abstract