DeepIG: Multi-Robot Information Gathering With Deep Reinforcement Learning

Abstract

State-of-the-art multi-robot information gathering (MR-IG) algorithms often rely on a model that describes the structure of the information of interest to drive the robots motion. This causes MR-IG algorithms to fail when they are applied to new IG tasks, as existing models cannot describe the information of interest. Therefore, we propose in this paper a MR-IG algorithm that can be applied to new IG tasks with little algorithmic changes. To this end, we introduce DeepIG: a MR-IG algorithm that uses Deep Reinforcement Learning to allow robots to learn how to gather information. Nevertheless, there are IG tasks for which accurate models have been derived. Therefore, we extend DeepIG to exploit existing models for such IG tasks. This algorithm we term it model-based DeepIG (MB-DeepIG). First, we evaluate DeepIG in simulations, and in an indoor experiment with three quadcopters that autonomously map an unknown terrain profile built in our lab. Results demonstrate that DeepIG can be applied to different IG tasks without algorithmic changes, and that it is robust to measurement noise. Then, we benchmark MB-DeepIG against state-of-the-art informationdriven Gaussian-processes-based IG algorithms. Results demonstrate that MB-DeepIG outperforms the considered benchmarks.

Robotic Information Gathering With Reinforcement Learning Assisted by Domain Knowledge: An Application to Gas Source Localization

“…Robotic IG has been widely researched in the context of multiple applications such as exploration [7], robot navigation [8], [9] tracking and surveillance [10].…”

“…In practice, Reinforcement learning (RL) is a promising solution to derive flexible strategies for IG. In [7] the authors developed a Deep-RL IG algorithm that outperforms state-ofthe-art Gaussian-Processes-based benchmarks. The authors in [7] use model-free RL.…”

“…In [7] the authors developed a Deep-RL IG algorithm that outperforms state-ofthe-art Gaussian-Processes-based benchmarks. The authors in [7] use model-free RL. This has a definite advantage: it does not make any assumption about the underlying physical process.…”

“…Thus, the IG algorithm exploits the domain knowledge offered by the model to derive intelligent strategies. Examples of models used in IG include, but are not limited to, Gaussian processes [7], [10], partially observable Markov decision processes [17] and partial differential equations (PDEs) [18], [19]. Model-based IG achieves a superior performance in tasks for which the model accurately describes the process of interest.…”

Alpha C2–An Intelligent Air Defense Commander Independent of Human Decision-Making

“…Centralized command has higher operational efficiency and is the current mainstream command mode. Moreover, with a large number of combat units, the use of a multi-intelligent agent structure [31] will incur a great communication burden. Therefore, a command structure is adopted in this study in which an intelligent agent controls multiple fire units to make air defense command decisions.…”