Three-Dimensional Path Following Control of an Underactuated Robotic Dolphin Using Deep Reinforcement Learning

Liu, Jincun; Liu, Zhenna; Wu, Zhengxing; Yu, Junzhi

doi:10.1109/rcar49640.2020.9303309

Cited by 10 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Taking into account both attitude control and position control, 2D or 3D path tracking is a common motion control task for bionic underwater robots [ 98 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 128 , 129 ] that enables the motion capability of bionic robots in underwater environments. The authors of [ 111 ] deployed the DDQN algorithm to the path tracking control of a hybrid-driven robotic fish, and quantitatively compared the control performance of RL with PID and SMC.…”

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

Self Cite

View full text Add to dashboard Cite

Bionic robots possess inherent advantages for underwater operations, and research on motion control and intelligent decision making has expanded their application scope. In recent years, the application of reinforcement learning algorithms in the field of bionic underwater robots has gained considerable attention, and continues to grow. In this paper, we present a comprehensive survey of the accomplishments of reinforcement learning algorithms in the field of bionic underwater robots. Firstly, we classify existing reinforcement learning methods and introduce control tasks and decision making tasks based on the composition of bionic underwater robots. We further discuss the advantages and challenges of reinforcement learning for bionic robots in underwater environments. Secondly, we review the establishment of existing reinforcement learning algorithms for bionic underwater robots from different task perspectives. Thirdly, we explore the existing training and deployment solutions of reinforcement learning algorithms for bionic underwater robots, focusing on the challenges posed by complex underwater environments and underactuated bionic robots. Finally, the limitations and future development directions of reinforcement learning in the field of bionic underwater robots are discussed. This survey provides a foundation for exploring reinforcement learning control and decision making methods for bionic underwater robots, and provides insights for future research.

show abstract

Section: Rl-based Methods In Task Spaces Of Bionic Underwater Robotsmentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Robotic fish motion control methods can be categorized into fishlike gait generation control 13,14,[16][17][18][19] and target reaching motion control. [20][21][22] The former category produces a swimming pattern like a real fish but does not have the capability to reach the target. The latter category concentrates on controlling the motion of an entire body that includes feedback to achieve the desired motion to reach the target.…”

Section: Introductionmentioning

confidence: 99%

Real-time implementation of deep reinforcement learning controller for speed tracking of robotic fish through data-assisted modeling

Duraisamy

Santhanakrishnan

Amirtharajan

et al. 2023

Proceedings of the Institution of Mechanical Engineers, Part C:

View full text Add to dashboard Cite

This article proposes real-time speed tracking of two-link surface swimming robotic fish using a deep reinforcement learning (DRL) controller. Hydrodynamic modelling of robotic fish is done by virtue of Newtonian dynamics and Lighthill’s kinematic model. However, this includes external unsteady reactive forces that cannot be modeled accurately due to the distributed nature of hydrodynamic behavior. Therefore, a novel data-assisted dynamic model and control method is proposed for the speed tracking of robotic fish. Initially, the cruise speed motion data are collected through experiments. The water-resistance coefficient is estimated using the least mean square fit, which is then adopted in the model. Subsequently, a closed-loop discrete-time DRL controller trained through a soft actor-critic (SAC) agent is implemented through simulations. SAC overcomes the brittleness problem encountered by other policy gradient approaches by encouraging the policy network for maximum exploration and not assigning a higher probability to any single part of actions. Due to this robustness in the policy learning, the convergence error becomes low in RL-SAC than RL-DDPG controller. The simulation results verify that the DRL-SAC control with data-assisted modelling substantially improves the speed tracking performance. Further, this controller is validated in real-time, and it is observed that the SAC-trained controller tracks the desired speed more accurately than the DDPG controller.

show abstract