Markov Chain Approach to Probabilistic Guidance for Swarms of Autonomous Agents

Abstract

Motivated by biological swarms occurring in nature, there is recent interest in developing swarms comprised completely of engineered agents. The main challenge for developing swarm guidance laws compared to earlier formation flying and multi‐vehicle coordination approaches is the sheer number of agents involved. While formation flying applications might involve up to 10 to 20 agents, swarms are desired to contain hundreds to many thousands of agents. In order to deal with the sheer size, the present paper makes a break with past deterministic methods, and considers the swarm as a statistical ensemble for which guidance can be performed from a probabilistic point of view. The probability‐based approach takes advantage of the law of large numbers, and leads to computationally tractable and implementable swarm guidance laws. Agents following a probabilistic guidance algorithm make statistically independent probabilistic decisions based solely on their own state, which ultimately guides the swarm to the desired density distribution in the configuration space. Two different synthesis methods are introduced for designing probabilistic guidance laws. The first is based on the Metropolis‐Hastings (M‐H) algorithm, and the second is based on using linear matrix inequalities (LMIs). The M‐H approach ensures convergent swarm behavior subject to enforced desired motion constraints, while the LMI approach additionally ensures exponential convergence with a prescribed decay rate, and allows minimization of a cost function that reflects fuel expenditure. In addition, both algorithms endow the swarm with the property of self‐repair, and the capability to strictly enforce zero‐probability keep‐out regions. This last property requires a slight generalization of the Perron‐Frobenius theory, and can be very useful in swarm applications that contain regions where no agents are allowed to go. Simulation examples are given to illustrate the methods and demonstrate desired properties of the guided swarm.

Markov Chains With Maximum Return Time Entropy for Robotic Surveillance

“…where the third line uses (1). Regarding (ii), when the graph is complete and P = 1 n π , the return time T ii follows the geometric distribution:…”

“…Stochastic surveillance strategies, which emphasize the unpredictability of the movement of the patroller, are desirable since they are capable of defending against intelligent intruders who aim to avoid detection/capture. One of the main approaches to the design of robotic stochastic surveillance strategies is to adopt Markov chains; e.g., see the early reference [14] and the more recent [1], [6], [9], [19]. Srivastava et al [24] justified the Markov chain-based stochastic surveillance strategy by showing that for the deterministic strategies, in addition to predictability, it is also hard to specify the visit frequency.…”

Maximal Invariant Set Computation and Design for Markov Chains

“…These two assumptions ensure that there exists some Markov matrix M for which lim k→∞ M k = v1 T , and that Algorithm 2 terminates as a result of Proposition III.6 from the existence of some ε > 0 such that g − Gv ≥ ε1. Furthermore if A a is symmetric, then an ergodic, reversible Markov chain can be explicitly constructed, e.g., with the Metropolis-Hastings algorithm, which would ensure that a feasible solution exists [12].…”

“…If no reversible ergodic Markov matrix is found, then this problem can be modified by replacing (13e) with LMI (11) and P ≥ 0. Bilinearity can be avoided by fixing D, e.g., D = diag(v) −1 as suggested in [12], and testing various values of λ in the interval [0, 1] to find the minimum. This procedure requires solving a sequence of linear feasibility problems in the matrix variable M .…”

Distributed Control for Spatial Self-Organization of Multi-agent Swarms

“…The localization of each agent still remains to be a main assumption. Under similar conditions, one can find the manuscripts [1] and [8], which describe probabilistic swarm guidance algorithms. In [5], the authors present an approach to task allocation for a homogeneous swarm of robots.…”