Path planning and dynamic collision avoidance algorithm under COLREGs via deep reinforcement learning

Xu, Xiaoying; Cai, Peng; Ahmed, Zahoor; Yellapu, Vidya Sagar; Zhang, Weidong

doi:10.1016/j.neucom.2021.09.071

Cited by 53 publications

(20 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The purpose of path planning is to reach the target point, but previous research has mainly focused on reaching a single goal in dynamic or unpredictable environments. [3], [11], [16], [17], [42], [68]. The goal should be pre-defined by the user and the agent who completed learning can reach only the defined goal.…”

Section: A Advantage Of the Proposed Methodsmentioning

confidence: 99%

A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning

Lee

Kim

2023

IEEE Access

View full text Add to dashboard Cite

The aim of path planning is to search for a path from the starting point to the goal. Numerous studies, however, have dealt with a single predefined goal. That is, an agent who has completed learning cannot reach other goals that have not been visited in the training. In the present study, we propose a novel reinforcement learning (RL) framework for an agent reachable to any subgoal as well as the final goal in path planning. To do this, we utilize goal-conditioned RL and propose bidirectional memory editing to obtain various bidirectional trajectories of the agent. Bidirectional memory editing can generate various behavior and subgoals of the agent from the limited trajectory. Then, the generated subgoals and behaviors of the agent are trained on the policy network so that the agent can reach any subgoals from any starting point. In addition, we present reward shaping for the short path of the agent to reach the goal. In the experimental result, the agent was able to reach the various goals that had never been visited by the agent during the training. We confirmed that the agent could perform difficult missions, such as a round trip, and the agent used the shorter route with reward shaping.INDEX TERMS Controllable agent, path planning, goal-conditioned reinforcement learning, bidirectional memory editing.

show abstract

Section: A Advantage Of the Proposed Methodsmentioning

confidence: 99%

A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning

Lee

Kim

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…If non-uniform sampling is adopted, the learning rate al pha must be adjusted according to the sampling probability. Set the learning rate as Equation (10).…”

Section: The Improved Dqnmentioning

confidence: 99%

“…It was first proposed by Sutton, whose essence is to learn from the interaction. In recent years, there has been a boom in theoretical research and technical implementation of reinforcement learning, especially in the Deep Q-Network (DQN) algorithm [10,11] proposed by the Google team. In the shipping industry, reinforcement learning is widely used in collision avoidance.…”

Section: Introductionmentioning

confidence: 99%

Improved DQN for Dynamic Obstacle Avoidance and Ship Path Planning

Yang

Han

2023

Algorithms

View full text Add to dashboard Cite

The avoidance of collisions among ships requires addressing various factors such as perception, decision-making, and control. These factors pose many challenges for autonomous collision avoidance. Traditional collision avoidance methods have encountered significant difficulties when used in autonomous collision avoidance. They are challenged to cope with the changing environment and harsh motion constraints. In the actual navigation of ships, it is necessary to carry out decision-making and control under the constraints of ship manipulation and risk. From the implementation process perspective, it is a typical sequential anthropomorphic decision-making problem. In order to solve the sequential decision problem, this paper improves DQN by setting a priority for sample collection and adopting non-uniform sampling, and it is applied to realize the intelligent collision avoidance of ships. It also verifies the performance of the algorithm in the simulation environment.

show abstract

“…A well-suited machine learning approach to solve motion planning tasks in uncertain environments is RL [5], [6], [34], [35]. The regarded scenario is usually on the open sea with other non-reactive dynamic obstacles [6], [35] and static obstacles [5], [34].…”

Section: Related Workmentioning

confidence: 99%

Safe Reinforcement Learning for Urban Driving using Invariably Safe Braking Sets

Krasowski

Zhang

Althoff

2022

2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

Autonomous vehicles have to obey traffic rules. These rules are often formalized using temporal logic, resulting in constraints that are hard to solve using optimization-based motion planners. Reinforcement Learning (RL) is a promising method to find motion plans adhering to temporal logic specifications. However, vanilla RL algorithms are based on random exploration, which is inherently unsafe. To address this issue, we propose a provably safe RL approach that always complies with traffic rules. As a specific application area, we consider vessels on the open sea, which must adhere to the Convention on the International Regulations for Preventing Collisions at Sea (COLREGS). We introduce an efficient verification approach that determines the compliance of actions with respect to the COLREGS formalized using temporal logic. Our action verification is integrated into the RL process so that the agent only selects verified actions. In contrast to agents that only integrate the traffic rule information in the reward function, our provably safe agent always complies with the formalized rules in critical maritime traffic situations and, thus, never causes a collision.

show abstract

Path planning and dynamic collision avoidance algorithm under COLREGs via deep reinforcement learning

Cited by 53 publications

References 19 publications

A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning

A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning

Improved DQN for Dynamic Obstacle Avoidance and Ship Path Planning

Safe Reinforcement Learning for Urban Driving using Invariably Safe Braking Sets

Contact Info

Product

Resources

About