An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

The problem of optimal trajectory planning of the unmanned underwater vehicle (UUV) is considered and analytically solved. The task is to minimize the risk of detection of the moving object by a static sonar while moving between two given points on a plane. The detection is based on the primary acoustic field radiated by the object with a non-uniform radiation pattern. In the first part of the article, the probability of non-detection is derived. Further, it is used as an optimization criterion. The non-uniform radiation pattern of the object differentiates this work from previous research in the area. The optimal trajectory and velocity law of the moving object are found, as well as the criterion value on it.

show abstract

“…Proof. Choose the non-detection probability of the form Equation (10) and consider it at a small signal/noise ratio…”

Section: Lemmamentioning

confidence: 99%

“…Some of them may be functionally related. In References [10,11], the path planning problem criterion is based on the artificial potential field which is formed by the obstacles. In Reference [12], optimal path consists of lines and circular arcs (2D Dubins curves).…”

mentioning

confidence: 99%

Path Planning in Threat Environment for UUV with Non-Uniform Radiation Pattern

Galyaev

Dobrovidov

Lysenko

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The route planning problem is often addressed in RL, utilizing an RL algorithm that yields the maximum reward value for an unmanned ship. In aircraft detection systems and radar designs, RL is applied to optimal radar system design and aircraft image analysis to detect radio waves and minimize unnecessary interference [25][26][27].…”

Section: Learning From Demonstrationmentioning

confidence: 99%

Adaptive Human–Machine Evaluation Framework Using Stochastic Gradient Descent-Based Reinforcement Learning for Dynamic Competing Network

Kim

Lee

2020

Applied Sciences

View full text Add to dashboard Cite

Complex problems require considerable work, extensive computation, and the development of effective solution methods. Recently, physical hardware- and software-based technologies have been utilized to support problem solving with computers. However, problem solving often involves human expertise and guidance. In these cases, accurate human evaluations and diagnoses must be communicated to the system, which should be done using a series of real numbers. In previous studies, only binary numbers have been used for this purpose. Hence, to achieve this objective, this paper proposes a new method of learning complex network topologies that coexist and compete in the same environment and interfere with the learning objectives of the others. Considering the special problem of reinforcement learning in an environment in which multiple network topologies coexist, we propose a policy that properly computes and updates the rewards derived from quantitative human evaluation and computes together with the rewards of the system. The rewards derived from the quantitative human evaluation are designed to be updated quickly and easily in an adaptive manner. Our new framework was applied to a basketball game for validation and demonstrated greater effectiveness than the existing methods.

show abstract

“…Due to this reason, an effective path planning algorithm for multi-arm manipulators has to be developed. In the literature about path planning, there are already some deep learning-based approaches implemented for robot applications such as mobile manipulation [ 20 , 21 ], unmanned ship [ 22 ] and even for multi-mobile robot [ 23 ]. These imply that deep learning-based approach can be promising in path planning for single arm manipulator [ 24 ] and also for multi-arm manipulators [ 25 ].…”

Section: Introductionmentioning

confidence: 99%

Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor–Critic with Hindsight Experience Replay

Prianto

Kim

Park

et al. 2020

Sensors

View full text Add to dashboard Cite

Since path planning for multi-arm manipulators is a complicated high-dimensional problem, effective and fast path generation is not easy for the arbitrarily given start and goal locations of the end effector. Especially, when it comes to deep reinforcement learning-based path planning, high-dimensionality makes it difficult for existing reinforcement learning-based methods to have efficient exploration which is crucial for successful training. The recently proposed soft actor–critic (SAC) is well known to have good exploration ability due to the use of the entropy term in the objective function. Motivated by this, in this paper, a SAC-based path planning algorithm is proposed. The hindsight experience replay (HER) is also employed for sample efficiency and configuration space augmentation is used in order to deal with complicated configuration space of the multi-arms. To show the effectiveness of the proposed algorithm, both simulation and experiment results are given. By comparing with existing results, it is demonstrated that the proposed method outperforms the existing results.

show abstract

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

Cited by 153 publications

References 31 publications

Path Planning in Threat Environment for UUV with Non-Uniform Radiation Pattern

Path Planning in Threat Environment for UUV with Non-Uniform Radiation Pattern

Adaptive Human–Machine Evaluation Framework Using Stochastic Gradient Descent-Based Reinforcement Learning for Dynamic Competing Network

Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor–Critic with Hindsight Experience Replay

Contact Info

Product

Resources

About