On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator

Gradient-based methods have been widely used for system design and optimization in diverse application domains. Recently, there has been a renewed interest in studying theoretical properties of these methods in the context of control and reinforcement learning. This article surveys some of the recent developments on policy optimization, a gradient-based iterative approach for feedback control synthesis that has been popularized by successes of reinforcement learning. We take an interdisciplinary perspective in our exposition that connects control theory, reinforcement learning, and large-scale optimization. We review a number of recently developed theoretical results on the optimization landscape, global convergence, and sample complexityof gradient-based methods for various continuous control problems, such as the linear quadratic regulator (LQR), [Formula: see text] control, risk-sensitive control, linear quadratic Gaussian (LQG) control, and output feedback synthesis. In conjunction with these optimization results, we also discuss how direct policy optimization handles stability and robustness concerns in learning-based control, two main desiderata in control engineering. We conclude the survey by pointing out several challenges and opportunities at the intersection of learning and control.

show abstract

Survey of recent progress in data-driven policy optimization for controller design

Zhao¹,

You²

2023

Sci. Sin.-Inf.

View full text Add to dashboard Cite

On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator

Cited by 3 publications

References 24 publications

Escaping High-order Saddles in Policy Optimization for Linear Quadratic Gaussian (LQG) Control

Escaping High-order Saddles in Policy Optimization for Linear Quadratic Gaussian (LQG) Control

Toward a Theoretical Foundation of Policy Optimization for Learning Control Policies

Survey of recent progress in data-driven policy optimization for controller design

Contact Info

Product

Resources

About