Connectivity Properties of the Set of Stabilizing Static Decentralized Controllers

Han, Feng; Lavaei, Javad

doi:10.1137/19m123765x

Cited by 16 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, it is shown in [29] that a class of finite-horizon output-feedback linear quadratic control problems also satisfies the gradient dominance property. Some recent studies have examined the connectivity of stabilizing static output feedback policies [20,13,30]. It is shown in [20] that the set of stabilizing static output feedback policies can be highly disconnected, which poses a significant challenge for decentralized LQR problems.…”

Section: Related Workmentioning

confidence: 99%

“…This makes its optimization landscape richer and yet much more complicated than LQR. Indeed, the set of stabilizing static state feedback policies is connected, but the set of stabilizing static output feedback policies can be highly disconnected [20]. The connectivity of stabilizing dynamical output feedback policies, i.e., the feasible region of LQG control, remains unclear.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

Tang¹,

Zheng²,

Li³

2021

Preprint

View full text Add to dashboard Cite

This paper revisits the classical Linear Quadratic Gaussian (LQG) control from a modern optimization perspective. We analyze two aspects of the optimization landscape of the LQG problem: 1) connectivity of the set of stabilizing controllers C n ; and 2) structure of stationary points. It is known that similarity transformations do not change the input-output behavior of a dynamical controller or LQG cost. This inherent symmetry by similarity transformations makes the landscape of LQG very rich. We show that 1) the set of stabilizing controllers C n has at most two path-connected components and they are diffeomorphic under a mapping defined by a similarity transformation; 2) there might exist many strictly suboptimal stationary points of the LQG cost function over C n and these stationary points are always non-minimal ; 3) all minimal stationary points are globally optimal and they are identical up to a similarity transformation. These results shed some light on the performance analysis of direct policy gradient methods for solving the LQG problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

Tang¹,

Zheng²,

Li³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this case, a static outputfeedback (SOF) gain u t = Ky t with K ∈ R m×d is typically insufficient to obtain good control performance. In fact, the set of stabilizing SOF gains can be highly disconnected [20], and even finding a stabilizing SOF controller is generally a challenging task [22], [23]. Unlike SOF control, under Assumption 1, a stabilizing dynamic output controller always exists and can be found easily, thanks to the well-known separation principle [27].…”

Section: A Linear Quadratic Controlmentioning

confidence: 99%

“…Unlike state-feedback LQR problems, it is shown that policy gradient methods are unlikely to find the globally optimal SOF controller. This is because the set of stabilizing SOF controllers is typically disconnected, and stationary points can be local minima, saddle points, or even local maxima [10], [20]. Moreover, even finding a stabilizing SOF controller is a challenging task [22], [23].…”

Section: Introductionmentioning

confidence: 99%

On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

Duan¹,

Cao²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

The optimization landscape of optimal control problems plays an important role in the convergence of many policy gradient methods. Unlike state-feedback Linear Quadratic Regulator (LQR), static output-feedback policies are typically insufficient to achieve good closed-loop control performance. We investigate the optimization landscape of linear quadratic control using dynamic output-feedback policies, denoted as dynamic LQR (dLQR) in this paper. We first show that the dLQR cost varies with similarity transformations. We then derive an explicit form of the optimal similarity transformation for a given observable stabilizing controller. We further characterize the unique observable stationary point of dLQR. This provides an optimality certificate for policy gradient methods under mild assumptions. Finally, we discuss the differences and connections between dLQR and the canonical linear quadratic Gaussian (LQG) control. These results shed light on designing policy gradient algorithms for decisionmaking problems with partially observed information.

show abstract

“…Noted that state feedback LQR can be regarded as a special case of SOF. Compared with state feedback LQR, the domain of the output control gain of SOF can be disconnected, while stationary points in each component can be local minima, saddle points, or even local maxima [12], [21]- [23]. Fatkhullin and Polyak (2020) focused on continuous-time SOF and analyzed the convergence rate to stationary points [12].…”

Section: Introductionmentioning

confidence: 99%

Optimization Landscape of Gradient Descent for Discrete-time Static Output Feedback

Duan

Zhao

2021

Preprint

View full text Add to dashboard Cite

In this paper, we analyze the optimization landscape of the gradient descent method for static output feedback (SOF) control of discrete-time linear time-invariant systems with quadratic cost. The SOF setting can be quite common, for example, when there are unmodeled hidden states in the underlying process. We first identify several important properties of the SOF cost function, including coercivity, L-smoothness, and M -Lipschitz continuous Hessian. Based on these results, we show that when the observation matrix has full column rank, gradient descent is able to converge to the global optimal controller at a linear rate. For the partially observed case, convergence to stationary points is obtained in general and the corresponding convergence rate is characterized. In the latter more challenging case, we further prove that under some mild conditions, gradient descent converges linearly to a local minimum if the starting point is close to one. These results not only characterize the performance of gradient descent in optimizing the SOF problem, but also shed light on the efficiency of general policy gradient methods in reinforcement learning.

show abstract

Connectivity Properties of the Set of Stabilizing Static Decentralized Controllers

Cited by 16 publications

References 21 publications

Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

Optimization Landscape of Gradient Descent for Discrete-time Static Output Feedback

Contact Info

Product

Resources

About