An AUV Target-Tracking Method Combining Imitation Learning and Deep Reinforcement Learning

Mao, Yubing; Gao, Farong; Zhang, Qizhong; Yang, Zhangyi

doi:10.3390/jmse10030383

Cited by 17 publications

(12 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, four strategies, i.e., prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, were applied in [ 106 ], respectively, to eliminate the correlation of training data, ensure the stability and speed of the convergence of the reinforcement learning AC network, update the critic network faster, and more accurately evaluate and improve the actor network’s generalization ability. Highly correlated data may lead to local convergence in RL [ 133 ]. One solution is to perform random sampling in the experience replay buffer, but this solution is only suitable for off-policy RL [ 133 ].…”

Section: Training and Deployment Methods Of Rl On Bionic Underwater R...mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

View full text Add to dashboard Cite

Bionic robots possess inherent advantages for underwater operations, and research on motion control and intelligent decision making has expanded their application scope. In recent years, the application of reinforcement learning algorithms in the field of bionic underwater robots has gained considerable attention, and continues to grow. In this paper, we present a comprehensive survey of the accomplishments of reinforcement learning algorithms in the field of bionic underwater robots. Firstly, we classify existing reinforcement learning methods and introduce control tasks and decision making tasks based on the composition of bionic underwater robots. We further discuss the advantages and challenges of reinforcement learning for bionic robots in underwater environments. Secondly, we review the establishment of existing reinforcement learning algorithms for bionic underwater robots from different task perspectives. Thirdly, we explore the existing training and deployment solutions of reinforcement learning algorithms for bionic underwater robots, focusing on the challenges posed by complex underwater environments and underactuated bionic robots. Finally, the limitations and future development directions of reinforcement learning in the field of bionic underwater robots are discussed. This survey provides a foundation for exploring reinforcement learning control and decision making methods for bionic underwater robots, and provides insights for future research.

show abstract

Section: Training and Deployment Methods Of Rl On Bionic Underwater R...mentioning

confidence: 99%

“…Highly correlated data may lead to local convergence in RL [ 133 ]. One solution is to perform random sampling in the experience replay buffer, but this solution is only suitable for off-policy RL [ 133 ]. Another solution is multi-agent RL.…”

Section: Training and Deployment Methods Of Rl On Bionic Underwater R...mentioning

confidence: 99%

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Feng

Wang

et al. 2023

Biomimetics

View full text Add to dashboard Cite

show abstract

“…The calculation equation of uncertainty is: The target probability map represents the probability of the existence of a target in a 3D environment [17]. The target probability map of the 3D environment is defined in Equation (5).…”

Section: Uncertainty Mapmentioning

confidence: 99%

“…The existence of fish and glaciers in the marine environment make it difficult for AUVs to carry out underwater unmanned operation, while an AUV's own energy consumption and communication restriction also limit the efficiency of underwater missions. Research on how to realize the target search and complete the corresponding tasks scientifically and efficiently in the complex 3D environment has been gaining popularity among scholars [3][4][5].…”

Section: Introductionmentioning

confidence: 99%

Improved RRT Algorithm for AUV Target Search in Unknown 3D Environment

Chen

et al. 2022

JMSE

View full text Add to dashboard Cite

Due to the complexity of the marine environment, underwater target search and interception is one of the biggest problems faced by an autonomous underwater vehicle (AUV). At present, there is quite a lot of research in terms of the two-dimensional environment. This paper proposes an improved rapidly exploring random trees (RRT) algorithm to solve the problem of target search and interception in an unknown three-dimensional (3D) environment. The RRT algorithm is combined with rolling planning and node screening to realize path planning in an unknown environment, and then the improved RRT algorithm is applied to the search and interception process in a 3D environment. Combined with the search decision function and the three-point numerical differential prediction method, the RRT algorithm can search for and effectively intercept the target. Numerical simulations in various situations show the superior performance, in terms of time and accuracy, of the proposed approach.

show abstract

“…Autonomous underwater vehicles (AUVs) are widely used in military and commercial fields due to their small size, low cost, high degree of autonomy and flexible deployment. They are irreplaceable tools in applications such as accident rescue, seabed topographic mapping, object detection, observation of ocean phenomena and marine resource development [1][2][3][4][5][6][7][8]. With the increasing complexity of underwater tasks, it is necessary to improve work efficiency through the cooperation of multiple AUVs, which can even complete tasks that cannot be performed by a single AUV [9].…”

Section: Introductionmentioning

confidence: 99%

Formation Control of Multiple Autonomous Underwater Vehicles under Communication Delay, Packet Discreteness and Dropout

Zhang

et al. 2022

JMSE

View full text Add to dashboard Cite

Effective communication between multiple autonomous underwater vehicles (AUVs) is necessary for formation control. As the most reliable underwater communication method, acoustic communication still has many constraints compared with radio communication, which affects the effectiveness of formation control. Therefore, this paper proposes a formation control scheme for multiple AUVs under communication delay, packet discreteness and dropout. Firstly, the communication delay is estimated based on the kernel density estimation method. To solve the problem of packet discreteness and dropout, the curve fitting method is used to predict the states of the AUV. Secondly, a follower controller is designed based on the leader–follower approach using input–output feedback linearization, which is proven to be stable with Lyapunov stability theory. Then, some simulation results are presented to demonstrate the stability and accuracy of the formation control in different communication environments. Finally, the field tests on the lake show that the scheme introduced in this paper is valid and practical.

show abstract

An AUV Target-Tracking Method Combining Imitation Learning and Deep Reinforcement Learning

Cited by 17 publications

References 46 publications

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Improved RRT Algorithm for AUV Target Search in Unknown 3D Environment

Formation Control of Multiple Autonomous Underwater Vehicles under Communication Delay, Packet Discreteness and Dropout

Contact Info

Product

Resources

About