Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

Abstract

This paper revisits the classical Linear Quadratic Gaussian (LQG) control from a modern optimization perspective. We analyze two aspects of the optimization landscape of the LQG problem: 1) connectivity of the set of stabilizing controllers C n ; and 2) structure of stationary points. It is known that similarity transformations do not change the input-output behavior of a dynamical controller or LQG cost. This inherent symmetry by similarity transformations makes the landscape of LQG very rich. We show that 1) the set of stabilizing controllers C n has at most two path-connected components and they are diffeomorphic under a mapping defined by a similarity transformation; 2) there might exist many strictly suboptimal stationary points of the LQG cost function over C n and these stationary points are always non-minimal ; 3) all minimal stationary points are globally optimal and they are identical up to a similarity transformation. These results shed some light on the performance analysis of direct policy gradient methods for solving the LQG problem.

On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

“…Note that the controller parameterization in (4) does not explicitly rely on the knowledge of system parameters A, B, and C, which is more suitable to model-free policy learning settings than (3) [26]. In addition to A K , B K and C K , the transient behavior induced by initial controller states (also called initial state estimates) also plays a big role in the accumulated cost.…”

“…Unlike the vanilla LQR and the SOF that use static feedback policies [10], [19]- [21], the problem of dLQR searches over the set of dynamic controllers, which has rich yet complicated landscape properties. The recent work [25], [26] has analyzed the structure of optimal dynamic controllers for the classical Linear Quadratic Gaussian (LQG) control problem. It is found that all stationary points that correspond to minimal controllers (i.e., reachable and observable controllers) are globally optimal to LQG, and that these stationary points are identical up to similarity transformations [25], [26].…”

“…The recent work [25], [26] has analyzed the structure of optimal dynamic controllers for the classical Linear Quadratic Gaussian (LQG) control problem. It is found that all stationary points that correspond to minimal controllers (i.e., reachable and observable controllers) are globally optimal to LQG, and that these stationary points are identical up to similarity transformations [25], [26]. Different from the classical LQG that minimizes a limiting average cost (or the variance of the final steady state) [25], [26], the dLQR problem seeks to find a dynamic controller that minimizes an infinite-horizon accumulated cost.…”

“…It is found that all stationary points that correspond to minimal controllers (i.e., reachable and observable controllers) are globally optimal to LQG, and that these stationary points are identical up to similarity transformations [25], [26]. Different from the classical LQG that minimizes a limiting average cost (or the variance of the final steady state) [25], [26], the dLQR problem seeks to find a dynamic controller that minimizes an infinite-horizon accumulated cost. In this case, the system transient behavior induced by the initial system state and initial state estimate should be considered.…”

“…This means that the initial state of the system is not negligible, and the symmetry induced by similarity transformations may not hold for dLQR. Therefore, recent results of LQG [25], [26] are not directly applicable to this dLQR problem.…”