Decentralized control problems with substitutable actions

Abstract

We consider a decentralized system with multiple controllers and define substitutability of one controller by another in open-loop strategies. We explore the implications of this property on the optimization of closed-loop strategies. In particular, we focus on the decentralized LQG problem with substitutable actions. Even though the problem we formulate does not belong to the known classes of "simpler" decentralized problems such as partially nested or quadratically invariant problems, our results show that, under the substitutability assumption, linear strategies are optimal and we provide a complete state space characterization of optimal strategies. We also identify a family of information structures that all give the same optimal cost as the centralized information structure under the substitutability assumption. Our results suggest that open-loop substitutability can work as a counterpart of the information structure requirements that enable simplification of decentralized control problems.

Optimal Local and Remote Controllers With Unreliable Uplink Channels

“…. γ n t+1 ∈{0,1} n∈N LS(p n , γ n t+1 ) × V t+1 NB(γ n t+1 , α n t , x n t+1 ) n∈N n∈N α n t (dx n t+1 ) ≥ V t (θ 1:N t ).where the last inequality follows from the definition of the value function(39). This completes the proof of the induction step, and the proof of the theorem.August 21, 2018 DRAFT…”

“…Let's compute the right hand side of (39) in Theorem 1. We first consider V t+1 term on the right hand side in (39). From the induction hypothesis we have V t+1 NB(γ n t+1 , α n t , x n t+1 ) n∈N = QF P t+1 , vec µ NB(γ n t+1 , α n t , x n t+1 ) n∈N + n∈N tr P nn t+1 cov NB(γ n t+1 , α n t , x n t+1 ) + e t+1 = QF P t+1 , vec µ(α n t ) + γ n t+1 (x n t+1 − µ(α n t ))…”

“…Combining (85), (86), and (88), the right hand side of (84) is V t θ 1:N t from the definition of the value function (39)…”

Decentralized Control of Stochastically Switched Linear System with Unreliable Communication

“…To make this dependence apparent, we write α 1 t as ψ ⊳ t (z t , θ t ). Then, from (88) and using operators Π and Ψ defined in (29) and (31), (87) can be written as…”

Optimal local and remote controllers with unreliable communication

“…In general, linear control strategies are not optimal, and even the problem of finding the best linear control strategies is not convex [18]. Existing optimal solutions of two-controller decentralized problems require either specific information structures, such as static [19], partially nested [20][21][22][23][24][25], stochastically nested [26], or other specific properties, such as quadratic invariance [27] or substitutability [28]. None of the above properties hold in our problem due to either the unreliable communication or the nature of dynamics and cost function.…”