Low-Cost Multi-Agent Navigation via Reinforcement Learning With Multi-Fidelity Simulator

Abstract

In recent years, reinforcement learning (RL) has been widely used to solve multi-agent navigation tasks, and a high-fidelity level for the simulator is critical to narrow the gap between simulation and real-world tasks. However, high-fidelity simulators have high sampling costs and bottleneck the training model-free RL algorithms. Hence, we propose a Multi-Fidelity Simulator framework to train Multi-Agent Reinforcement Learning (MFS-MARL), reducing the total data cost with samples generated by a lowfidelity simulator. We apply the depth-first search to obtain local feasible policies on the low-fidelity simulator as expert policies to help the original reinforcement learning algorithm explore. We built a multivehicle simulator with variable fidelity levels to test the proposed method and compared it with the vanilla Soft Actor-Critic (SAC) and expert actor methods. The results show that our method can effectively obtain local feasible policies and can achieve a 23% cost reduction in multi-agent navigation tasks.

An Advanced Multi-Agent Reinforcement Learning Framework of Bridge Maintenance Policy Formulation

“…The multi-agent framework is adopted to solve the problem. Researchers in various fields have tried to extend the existing single-agent to multi-agent [24][25][26], such as Modular Q-Learning in which a single agent problem is divided into different subproblems, and each agent solves different subproblems, Ant Q-Learning of which all the agents share reward, and Nash Q-Learning which has greatly improved the efficiency of Q-Learning algorithms [27][28][29]. In this paper, the training process is completed by multi-agent parallel mode, and the optimal maintenance policy of the bridge is output by calculating the return of the whole structure.…”