Hierarchical Reinforcement Learning Framework Towards Multi-Agent Navigation

Ding, Wenhao; Li, Shuaijun; Qian, Huihuan; Chen, Yong Q.

doi:10.1109/robio.2018.8664803

Cited by 29 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To simplify the decision-making process, raw sensor data can be directly used as input. Ding et al [59] studied the hierarchical RL method using lidar data as input. A high-level evaluation module is responsible for perceiving the overall environmental risks, and a low-level control module is responsible for making action decisions.…”

Section: Developmentmentioning

confidence: 99%

Deep reinforcement learning based mobile robot navigation: A review

Zhu

Zhang

2021

Tsinghua Sci. Technol.

256

View full text Add to dashboard Cite

Navigation is a fundamental problem of mobile robots, for which Deep Reinforcement Learning (DRL) has received significant attention because of its strong representation and experience learning abilities. There is a growing trend of applying DRL to mobile robot navigation. In this paper, we review DRL methods and DRL-based navigation frameworks. Then we systematically compare and analyze the relationship and differences between four typical application scenarios: local obstacle avoidance, indoor navigation, multi-robot navigation, and social navigation. Next, we describe the development of DRL-based navigation. Last, we discuss the challenges and some possible solutions regarding DRL-based navigation.

show abstract

Section: Developmentmentioning

confidence: 99%

Deep reinforcement learning based mobile robot navigation: A review

Zhu

Zhang

2021

Tsinghua Sci. Technol.

256

View full text Add to dashboard Cite

show abstract

“…For example, Long et al [4] used the PPO algorithm end-to-end training obstacle avoidance algorithm on the Stage simulator. Ding et al [9] trained the DDPG and HMM joint algorithm on the Gazebo simulator [10] to solve the multi-vehicle collaborative path planning. Tai et al [11] trained ADDPG on V-REP [12] to solve the Mapless Navigation problem and Kahn et al [13] trained a model-based RL algorithm on open raves to solve obstacle avoidance.…”

Section: Related Workmentioning

confidence: 99%

Low-Cost Multi-Agent Navigation via Reinforcement Learning With Multi-Fidelity Simulator

Qiu

Liu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In recent years, reinforcement learning (RL) has been widely used to solve multi-agent navigation tasks, and a high-fidelity level for the simulator is critical to narrow the gap between simulation and real-world tasks. However, high-fidelity simulators have high sampling costs and bottleneck the training model-free RL algorithms. Hence, we propose a Multi-Fidelity Simulator framework to train Multi-Agent Reinforcement Learning (MFS-MARL), reducing the total data cost with samples generated by a lowfidelity simulator. We apply the depth-first search to obtain local feasible policies on the low-fidelity simulator as expert policies to help the original reinforcement learning algorithm explore. We built a multivehicle simulator with variable fidelity levels to test the proposed method and compared it with the vanilla Soft Actor-Critic (SAC) and expert actor methods. The results show that our method can effectively obtain local feasible policies and can achieve a 23% cost reduction in multi-agent navigation tasks.

show abstract

“…Both Tai et al (2018) and Jin et al (2019) have shown that systems similar to the ones above can also be used to enable navigation in the presence of human pedestrians using RL and geometric sensor inputs. Ding et al (2018) showed that a similar capability can also be achieved using RL to train a system to choose between target pursuit and collision avoidance by incorporating a Hidden Markov Model (HMM) (Stratonovich 1965) into a hierarchical model. Considering the specific case in which some humans in the scene can be assumed to be companions, Li et al (2018) showed that end-to-end learned approaches could also enable Socially Concomitant Navigation (SCN), i.e., navigation in which the robot not only needs to avoid collisions as in previous work, but also needs to maintain a sense of affinity with respect to the motion of its companion.…”

Section: Localmentioning

confidence: 99%

Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey

Xiao¹,

Liu²,

Warnell³

et al. 2020

Preprint

View full text Add to dashboard Cite

Moving in complex environments is an essential capability of intelligent mobile robots. Decades of research and engineering have been dedicated to developing sophisticated navigation systems to move mobile robots from one point to another. Despite their overall success, a recently emerging research thrust is devoted to developing machine learning techniques to address the same problem, based in large part on the success of deep learning. However, to date, there has not been much direct comparison between the classical and emerging paradigms to this problem. In this article, we survey recent works that apply machine learning for motion control in mobile robot navigation, within the context of classical navigation systems. The surveyed works are classified into different categories, which delineate the relationship of the learning approaches to classical methods. Based on this classification, we identify common challenges and promising future directions.

show abstract

Hierarchical Reinforcement Learning Framework Towards Multi-Agent Navigation

Cited by 29 publications

References 21 publications

Deep reinforcement learning based mobile robot navigation: A review

Deep reinforcement learning based mobile robot navigation: A review

Low-Cost Multi-Agent Navigation via Reinforcement Learning With Multi-Fidelity Simulator

Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey

Contact Info

Product

Resources

About