Passive dynamics in mean field control

Abstract

Abstract-Mean-field models are a popular tool in a variety of fields. They provide an understanding of the impact of interactions among a large number of particles or people or other "self-interested agents", and are an increasingly popular tool in distributed control.This paper considers a particular randomized distributed control architecture introduced in our own recent work. In numerical results it was found that the associated mean-field model had attractive properties for purposes of control. In particular, when viewed as an input-output system, its linearization was found to be minimum phase.In this paper we take a closer look at the control model. The results are summarized as follows:(i) The Markov Decision Process framework of Todorov is extended to continuous time models, in which the "control cost" is based on relative entropy. This is the basis of the construction of a family of Markovian generators, parameterized by a scalar ζ ∈ R. (ii) A decentralized control architecture is proposed in which each agent evolves as a controlled Markov process. A central authority broadcasts a common control signal {ζ t } to each agent. The central authority chooses {ζ t } based on an aggregate scalar output of the Markovian agents. This is the basis of the mean field model. (iii) Provided the control-free system (with ζ ≡ 0) is a reversible Markov process, the following identity holds for the transfer function G obtained from the linearization,where the right hand side denotes the power spectral density for the output of any one of the individual Markov processes (with ζ ≡ 0).

Ordinary Differential Equation Methods for Markov Decision Processes and Application to Kullback--Leibler Control Cost

“…Simultaneous computation of the optimal policies is essential in applications to "demand dispatch" for providing virtual energy storage from a large collection of flexible loads [1,16,7]. In these papers, randomized policies are designed for each of many thousands of electric loads in a distributed control architecture.…”