Convex Analysis in Decentralized Stochastic Control, Strategic Measures, and Optimal Solutions

Yüksel, Serdar; Saldi, Naci

doi:10.1137/15m1049129

Cited by 44 publications

(58 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under these conditions, the optimal solution of problem (9) can be computed via the solution of a convex program when the information structure is nonclassical. See [11], [15], [26], [27] for recent advances. The transitive closure of a directed graph can be efficiently computed using Warshall's algorithm [28].…”

Section: A Convex Information Relaxationmentioning

confidence: 99%

A Convex Information Relaxation for Constrained Decentralized Control Design Problems

Lin

Bitar

2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

We describe a convex programming approach to the calculation of lower bounds on the minimum cost of constrained decentralized control problems with nonclassical information structures. The class of problems we consider entail the decentralized output feedback control of a linear time-varying system over a finite horizon, subject to polyhedral constraints on the state and input trajectories, and sparsity constraints on the controller's information structure. As the determination of optimal control policies for such systems is known to be computationally intractable in general, considerable effort has been made in the literature to identify efficiently computable, albeit suboptimal, feasible control policies. The construction of computationally tractable bounds on their suboptimality is the primary motivation for the techniques developed in this note. Specifically, given a decentralized control problem with nonclassical information, we characterize an expansion of the given information structure, which ensures its partial nestedness, while maximizing the optimal value of the resulting decentralized control problem under the expanded information structure. The resulting decentralized control problem is cast as an infinitedimensional convex program, which is further relaxed via a partial dualization and restriction to affine dual control policies. The resulting problem is a finite-dimensional conic program whose optimal value is a provable lower bound on the minimum cost of the original constrained decentralized control problem.The last equality follows from the fact that e T 1 ξ = 1 P-almost surely. To show that W MZ T K 2 0, it suffices to show columnwise inclusion in the second-order cone, i.e.,where s i ∈ L 2 1 is the i th element of the random vector s. By definition, we have that W ξ K 2 0 for all ξ ∈ Ξ. Also, since s i ≥ 0 almost surely, we have that W (s i ξ) K 2 0 almost surely. It follows from the convexity of the second-order cone that W E[s i ξ] K 2 0. APPENDIX D PROOF OF LEMMA 6Define the vector r ∈ R |J| according toJune 5, 2019 DRAFT Define the matrix Ψ := P J M 1/2 , where M 1/2 is the unique square root of the symmetric positive definite matrix M. Note that the matrix M 1/2 is symmetric and positive definite (and hence invertible). It holds that r T P J MP T The second and the last equalities both follow from the fact that η J = Π J P ξ = P J ξ. The fourth equality follows from the definition of the matrix Ψ and the symmetry of the matrix M 1/2 . The fifth equality follows from the fact [33, Prop. 3.2] that Ψ T ΨΨ T † = Ψ † . The sixth equality follows from the fact [33, Prop. 3.1] that Ψ † ΨΨ T = Ψ It holds that E zη T = E E zη T η J = E zη T J L T J = r T P J MP T J L T J June 5, 2019 DRAFT

show abstract

Section: A Convex Information Relaxationmentioning

confidence: 99%

A Convex Information Relaxation for Constrained Decentralized Control Design Problems

Lin

Bitar

2019

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

show abstract

“…Remark 2. One common approach that is used in the literature [10], [11] to show the existence of team-optimal policies is strategic measure approach. In this approach, one first identifies a topology on the set of strategic measures Ξ := {Q γ : γ ∈ Γ} (in general, weak topology) and then proves the relative compactness of Ξ along with lower semicontinuity of the cost function J with respect to this topology.…”

Section: Existence Of the Optimal Strategy For Static Team Problemsmentioning

confidence: 99%

“…Then, if Ξ is closed with respect to this topology, then one can deduce the existence of an optimal policy via Weierstrass Extreme Value Theorem. The main problem in this approach is to prove the closeness of Ξ, because convergence with respect to the topology defined on Ξ does not in general preserve the statistical independence of the actions given the observations; that is, in the limiting strategic measure, action u i of Agent i may depend on observation y j of Agent j which is prohibited in the original problem (see, e.g., [11,Theorem 2.7]). Hence, to overcome this obstacle, in this paper we directly introduce a topology on the set policies Γ instead of the set of strategic measures Ξ.…”

Section: Existence Of the Optimal Strategy For Static Team Problemsmentioning

confidence: 99%

A Topology for Team Policies and Existence of Optimal Team Policies in Stochastic Team Theory

Saldi

2020

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

In this paper, we establish the existence of teamoptimal policies for static teams and a class of sequential dynamic teams. We first consider the static team problems and show the existence of optimal policies under certain regularity conditions on the observation channels by introducing a topology on the set of policies. Then we consider sequential dynamic teams and establish the existence of an optimal policy via the static reduction method of Witsenhausen. We apply our findings to the wellknown counterexample of Witsenhausen and the Gaussian relay channel problem.

show abstract

“…Preliminaries. We first introduce preliminaries following the presentation in [49], in particular, we introduce the characterizations laid out by Witsenhausen, through his Intrinsic Model [43]. Consider sequential systems and assume the action and measurement spaces are standard Borel spaces, that is, Borel subsets of complete, separable and metric spaces.…”

mentioning

confidence: 99%

“…The conditions above, however, are only sufficient conditions [49,Example 1]. We note however that as a Corollary for (ii) above, for the LQG setup, under partial nestedness, convexity in policies hold as a consequence of Radners theorem; we will study this case in Section 4.…”

mentioning

confidence: 99%

Convex Symmetric Stochastic Dynamic Teams and Their Mean-Field Limit

Sanjari

Yüksel

2019

2019 IEEE 58th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

This paper studies convex stochastic dynamic team problems with finite and infinite time horizons under decentralized information structures. First, we introduce two notions called exchangeable teams and symmetric information structures. We show that in convex exchangeable team problems an optimal policy exhibits a symmetry structure. We give a characterization for such symmetrically optimal teams for a general class of convex dynamic team problems under mild conditional independence conditions. In addition, through concentration of measure arguments, we establish the convergence of optimal policies for teams with N decision makers to the corresponding optimal policies for symmetric mean-field teams with infinitely many decision makers. As a by-product, we present an existence result for convex mean-field teams, where the main contribution of our paper is with respect to the information structure in the system when compared with the related results in the literature that have either assumed a classical information structure or a static information structure. We also apply these results to the important special case of LQG team problems, where while for partially nested LQG team problems with finite time horizons it is known that the optimal policies are linear, for infinite horizon problems the linearity of optimal policies has not been established in full generality. We also study average cost finite and infinite horizon dynamic team problems with a symmetric partially nested information structure and obtain globally optimal solutions where we establish linearity of optimal policies. Moreover, we also study average cost infinite horizon LQG dynamic teams under sparsity and delay constraints.

show abstract

Convex Analysis in Decentralized Stochastic Control, Strategic Measures, and Optimal Solutions

Cited by 44 publications

References 55 publications

A Convex Information Relaxation for Constrained Decentralized Control Design Problems

A Convex Information Relaxation for Constrained Decentralized Control Design Problems

A Topology for Team Policies and Existence of Optimal Team Policies in Stochastic Team Theory

Convex Symmetric Stochastic Dynamic Teams and Their Mean-Field Limit

Contact Info

Product

Resources

About