Integrating a Path Planner and an Adaptive Motion Controller for Navigation in Dynamic Environments

Abstract

Since an individual approach can hardly navigate robots through complex environments, we present a novel two-level hierarchical framework called JPS-IA3C (Jump Point Search improved Asynchronous Advantage Actor-Critic) in this paper for robot navigation in dynamic environments through continuous controlling signals. Its global planner JPS+ (P) is a variant of JPS (Jump Point Search), which efficiently computes an abstract path of neighboring jump points. These nodes, which are seen as subgoals, completely rid Deep Reinforcement Learning (DRL)-based controllers of notorious local minima. To satisfy the kinetic constraints and be adaptive to changing environments, we propose an improved A3C (IA3C) algorithm to learn the control policies of the robots’ local motion. Moreover, the combination of modified curriculum learning and reward shaping helps IA3C build a novel reward function framework to avoid learning inefficiency because of sparse reward. We additionally strengthen the robots’ temporal reasoning of the environments by a memory-based network. These improvements make the IA3C controller converge faster and become more adaptive to incomplete, noisy information caused by partial observability. Simulated experiments show that compared with existing methods, this JPS-IA3C hierarchy successfully outputs continuous commands to accomplish large-range navigation tasks at shorter paths and less time through reasonable subgoal selection and rational motions.

Deep Reinforcement Learning for Indoor Mobile Robot Path Planning

“…The results of each planning are different and might not be optimal. Our further work will improve this kind of non-optimal path planning algorithm with DRL [ 26 ] continuously. Moreover, we will also consider the path planning for multiple robots [ 43 , 44 ].…”

“…Francis et al [ 25 ] used PRM as the sampling-based planner, and AutoRL as the RL method in the indoor navigation context to realize long-range indoor navigation. Zeng et al [ 26 ] presented a jump point search improved asynchronous advantage Actor-Critic (JPS-IA3C) for robot navigation in dynamic environments, which computes an abstract path of neighboring jump points (sub goals) by the global planner JPS+ and learns the control policies of the robots’ local motion by the improved A3C (IA3C) algorithm. Here, the IA3C denotes the local planner combined with the global planner to realize path planner for a robot.…”

Special Issue on Mobile Robots Navigation

“…Zeng et al [8] present a two-level hierarchical framework for robot navigation in dynamic environments in a continuous way, named JPS-IA3C (Jump Point Search Improved Asynchronous Advantage Actor-Critic). On the one hand, the global planner JPS+ (P), which is a variant of JPS, efficiently computes a sequence of subgoals for the motion controller, which can eliminate first-move lag and avoid local minima.…”

A Novel Hierarchical Soft Actor-Critic Algorithm for Multi-Logistics Robots Task Allocation

“…A large amount of information will be generated during the operation of the intelligent storage system, which is characterized by the dynamic nature of order information, goods information and storage information. Therefore, a large number of warehousing logistics robots and artificial intelligence technologies are needed to optimize decision-making [4][5][6]. The dynamic task allocation [7] problem of orders belongs to a part of picking work, which includes the process of orders batch, orders task allocation, path planning, picking, packing and shipping.…”