Multi-agent Control and Optimization under Uncertainty

Abstract

Many natural and engineering systems can be characterized as a collection of interacting agents each having access to local information, making local decisions, having local interactions with neighbors, and seeking to optimize local objectives. The analysis and design of such systems falls under the broader framework of "multi-agent control and optimization".Here depending on the context, "agents" could be animals, autonomous vehicles, power plants, etc. Multi-agent systems have the potential for solving problems that are di cult or impossible for an individual agent or monolithic system to solve. However, the complexity associated with a potentially large number of interacting agents brings about challenging control and optimization problems for system designers and coordinators. Such di culty is often enhanced by the presence of noise/uncertainty that is pervasive in both biological and engineering systems.In this dissertation, we focus on the analysis and design of several multi-agent control/optimization algorithms for various problems under uncertainty. Swarming, flocking, schooling and other aggregations of organisms in groups have been studied extensively in biology (see [64,65,74]). Organisms in swarms can exploit several advantages of staying close to each other for more e↵ective foraging. For example, in [38] Grünbaum explains how social foragers more successfully perform chemotaxis over noisy gradients than individuals.Such biological advantage is also demonstrated in Passino [76] by modeling the behavior of E. coli and M. xanthus bacteria. In our first line of work, we develop a mathematical model for analyzing the benefits of social foraging in a noisy environment. We identify conditions i on the nutrient profile ensuring that local agent actions will lead to cohesive foraging. For convex, smooth nutrient profiles we formalize the way in which swarming for social foraging is better at handling the e↵ects of noise when compared to the average of individual foraging strategies. Under a swarming discipline, observational noise realizations that induce trajectories di↵ering too much from the group average are likely to be discarded because of each individual's need to maintain cohesion. As a result, the group trajectories are less a↵ected by noise. Simulation experiments indicate that our theoretical results are also robust to inter-agent communication constraints and non-convex nutrient profiles.The above results suggest that swarming-like approaches for the control and/or optimization of networked agents may provide an additional level of robustness. This is precisely the gist of our second line of work in which we consider a distributed computing algorithmic scheme for stochastic optimization which relies on modest communication requirements amongst processors and most importantly, does not require synchronization. Specifically, we analyze a scheme with N > 1 independent threads each implementing a stochastic gradient algorithm. The threads are coupled via a perturbation of the gradient (with...