Multi-Agent Reinforcement Learning for Visibility-based Persistent Monitoring

Chen, Jingxi; Baskaran, Amrish; Zhang, Zhongshun; Tokekar, Pratap

doi:10.1109/iros51168.2021.9635898

Cited by 10 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from monitoring an underlying vector field of interest [6], [7], which is typically approached as generalized coverage problem or as a variant of the orienteering problem [8], [9], our agent aims to maximize information gain in the vicinity of the targets (i.e., minimizes the uncertainty over the true target positions) [10], [11]. Since this uncertainty grows over time for targets not in direct view of the agent, the agent has to leave some of the targets temporarily untracked to relocate others and then return to those as soon as possible to avoid losing them.…”

Section: Introductionmentioning

confidence: 99%

Spatio-Temporal Self-Attention Network for Video Saliency Prediction

Wang

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

This work focuses on the persistent monitoring problem, where a set of targets moving based on an unknown model must be monitored by an autonomous mobile robot with a limited sensing range. To keep each target's position estimate as accurate as possible, the robot needs to adaptively plan its path to (re-)visit all the targets and update its belief from measurements collected along the way. In doing so, the main challenge is to strike a balance between exploitation, i.e., re-visiting previously-located targets, and exploration, i.e., finding new targets or re-acquiring lost ones. Encouraged by recent advances in deep reinforcement learning, we introduce an attention-based neural solution to the persistent monitoring problem, where the agent can learn the inter-dependencies between targets, i.e., their spatial and temporal correlations, conditioned on past measurements. This endows the agent with the ability to determine which target, time, and location to attend to across multiple scales, which we show also helps relax the usual limitations of a finite target set. We experimentally demonstrate that our method outperforms other baselines in terms of number of targets visits and average estimation error in complex environments. Finally, we implement and validate our model in a drone-based simulation experiment to monitor mobile ground targets in a high-fidelity simulator.

show abstract

Section: Introductionmentioning

confidence: 99%

Spatio-Temporal Self-Attention Network for Video Saliency Prediction

Wang

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…Recently, multi-agent reinforcement learning (MARL) methods have proven to be effective for policy searching in complex and high-dimensional multi-robot tasks and they have been explored for information gathering and coverage problems. Persistent monitoring and coverage with MARL involve defining equal utility for every state [12,13]. In the sampling problem, utilities are distributed unevenly, thus requiring nonuniform exploration.…”

Section: Introductionmentioning

confidence: 99%

MARLAS: Multi Agent Reinforcement Learning for cooperated Adaptive Sampling

Pan¹,

Manjanna²,

Kumar³

et al. 2022

Preprint

View full text Add to dashboard Cite

The multi-robot adaptive sampling problem aims at finding trajectories for a team of robots to efficiently sample the phenomenon of interest within a given endurance budget of the robots. In this paper, we propose a robust and scalable approach using decentralized Multi-Agent Reinforcement Learning for cooperated Adaptive Sampling (MARLAS) of quasi-static environmental processes. Given a prior on the field being sampled, the proposed method learns decentralized policies for a team of robots to sample high-utility regions within a fixed budget. The multirobot adaptive sampling problem requires the robots to coordinate with each other to avoid overlapping sampling trajectories. Therefore, we encode the estimates of neighbor positions and intermittent communication between robots into the learning process. We evaluated MARLAS over multiple performance metrics and found it to outperform other baseline multi-robot sampling techniques. We further demonstrate robustness to communication failures and scalability with both the size of the robot team and the size of the region being sampled. The experimental evaluations are conducted both in simulations on real data and in real robot experiments on demo environmental setup 1 .

show abstract