Constrained Environment Optimization for Prioritized Multi-Agent Navigation

Abstract

Traditional approaches for multi-agent navigation consider the environment as a fixed constraint, despite the obvious influence of spatial constraints on agents' performance. Yet hand-designing conducive environments is inefficient and potentially expensive. The goal of this paper is to consider the obstacle layout of the environment as a decision variable in a system-level optimization problem. In other words, we aim to find an automated solution that optimizes the obstacle layout to improve the performance of multi-agent navigation, under a variety of realistic constraints. Towards this end, we propose novel problems of unprioritized and prioritized environment optimization, where the former considers agents unbiasedly and the latter incorporates agent priorities into optimization. We show, through formal proofs, under which conditions the environment can change to guarantee completeness (i.e., all agents reach goals), and analyze the role of agent priorities in the environment optimization. We proceed to impose constraints on the environment optimization that correspond to real-world restrictions on obstacle changes, and formulate it mathematically as a constrained stochastic optimization problem. Since the relationship between agents, environment and performance is challenging to model, we leverage reinforcement learning to develop a model-free solution and a primal-dual mechanism to handle constraints. Distinct information processing architectures are integrated for various implementation scenarios, including online/offline optimization and discrete/continuous environment. Numerical results corroborate the theory and demonstrate the validity and adaptability of our approach.