On the Sample Complexity of the Linear Quadratic Regulator

Dean, Sarah; Mania, Horia; Matni, Nikolai; Recht, Benjamin; Tu, Stephen

doi:10.48550/arxiv.1710.01688

Cited by 82 publications

(210 citation statements)

References 36 publications

Supporting

Mentioning

205

Contrasting

Order By: Relevance

“…The above approaches offer mainly data-independent bounds which reveal how the state dimension n and other system theoretic parameters affect the sample complexity of system identification qualitatively. This is different from finite sample data-dependent bounds-see for example bootstrapping [8] or [26], which might be more tight and more suitable for applications but do not necessarily reveal this dependence.…”

Section: Introductionmentioning

confidence: 85%

“…With the advances in high-dimensional statistics [6], there has been a recent shift from asymptotic analysis with infinite data to statistical analysis of system identification with finite samples. Over the past two years there have been significant advances in understanding finite sample system identification for both fully-observed systems [7][8][9][10][11][12][13][14] as well as partially-observed systems [15][16][17][18][19][20][21][22][23][24]. A tutorial can be found in [25].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Linear Systems can be Hard to Learn

Tsiamis¹,

Pappas²

2021

Preprint

View full text Add to dashboard Cite

In this paper, we investigate when system identification is statistically easy or hard, in the finite sample regime. Statistically easy to learn linear system classes have sample complexity that is polynomial with the system dimension. Most prior research in the finite sample regime falls in this category, focusing on systems that are directly excited by process noise. Statistically hard to learn linear system classes have worst-case sample complexity that is at least exponential with the system dimension, regardless of the identification algorithm. Using tools from minimax theory, we show that classes of linear systems can be hard to learn. Such classes include, for example, under-actuated or under-excited systems with weak coupling among the states. Having classified some systems as easy or hard to learn, a natural question arises as to what system properties fundamentally affect the hardness of system identifiability. Towards this direction, we characterize how the controllability index of linear systems affects the sample complexity of identification. More specifically, we show that the sample complexity of robustly controllable linear systems is upper bounded by an exponential function of the controllability index. This implies that identification is easy for classes of linear systems with small controllability index and potentially hard if the controllability index is large. Our analysis is based on recent statistical tools for finite sample analysis of system identification as well as a novel lower bound that relates controllability index with the least singular value of the controllability Gramian.

show abstract

Section: Introductionmentioning

confidence: 85%

Section: Introductionmentioning

confidence: 99%

Linear Systems can be Hard to Learn

Tsiamis¹,

Pappas²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This paper summarizes, and in some places extends, our recent theoretical and computational contributions [43,44,45,46,47,48,49,50,51,52,53,54,55,19] to the area of constrained optimal and robust controller synthesis. In particular, we…”

Section: Main Contributionsmentioning

confidence: 70%

“…In particular, we know of no results in the literature that bound the degradation in performance of controlling an uncertain system in terms of the size of the perturbations affecting it. While traditional results in robust control considered fixed model uncertainty, in the data-driven and learning based control setting, analytic interpretability of the effects uncertainty size are essential in obtaining sample-complexity bounds for these methods [19,20,21,22].…”

Section: Robust Controlmentioning

confidence: 99%

System Level Synthesis

Anderson¹,

Doyle²,

Low³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

This article surveys the System Level Synthesis framework, which presents a novel perspective on constrained robust and optimal controller synthesis for linear systems. We show how SLS shifts the controller synthesis task from the design of a controller to the design of the entire closed loop system, and highlight the benefits of this approach in terms of scalability and transparency. We emphasize two particular applications of SLS, namely large-scale distributed optimal control and robust control. In the case of distributed control, we show how SLS allows for localized controllers to be computed, extending robust and optimal control methods to largescale systems under practical and realistic assumptions. In the case of robust control, we show how SLS allows for novel design methodologies that, for the first time, quantify the degradation in performance of a robust controller due to model uncertainty -such transparency is key in allowing robust control methods to interact, in a principled way, with modern techniques from machine learning and statistical inference. Throughout, we emphasize practical and efficient computational solutions, and demonstrate our methods on easy to understand case studies.

show abstract

“…Machine learning has produced considerable recent interest in finite-time performance guarantees for system identification and control. In control, results have focused on finite time regret bounds for the LQR problem with unknown dynamics Dean et al, 2017Dean et al, , 2018Mania et al, 2019;Dean et al, 2019;Cohen et al, 2019), with Simchowitz & Foster (2020) ultimately settling the minimax optimal regret in terms of dimension and time horizon; others have considered regret in online adversarial settings (Agarwal et al, 2019;. Recent results in system identification have focused on obtaining finite time high probability bounds on the estimation error of the system's parameters when observing the evolution over time (Tu et al, 2017;Faradonbeh et al, 2018;Hazan et al, 2018;Hardt et al, 2018;Simchowitz et al, 2018;Sarkar & Rakhlin, 2018;Oymak & Ozay, 2019;Simchowitz et al, 2019;Tsiamis & Pappas, 2019).…”

Section: Related Workmentioning

confidence: 99%

Task-Optimal Exploration in Linear Dynamical Systems

Wagenmaker,

Simchowitz,

Jamieson

2021

Preprint

View full text Add to dashboard Cite

Exploration in unknown environments is a fundamental problem in reinforcement learning and control. In this work, we study task-guided exploration and determine what precisely an agent must learn about their environment in order to complete a particular task. Formally, we study a broad class of decision-making problems in the setting of linear dynamical systems, a class that includes the linear quadratic regulator problem. We provide instance-and taskdependent lower bounds which explicitly quantify the difficulty of completing a task of interest. Motivated by our lower bound, we propose a computationally efficient experiment-design based exploration algorithm. We show that it optimally explores the environment, collecting precisely the information needed to complete the task, and provide finite-time bounds guaranteeing that it achieves the instance-and task-optimal sample complexity, up to constant factors. Through several examples of the LQR problem, we show that performing task-guided exploration provably improves on exploration schemes which do not take into account the task of interest. Along the way, we establish that certainty equivalence decision making is instance-and task-optimal, and obtain the first algorithm for the linear quadratic regulator problem which is instance-optimal. We conclude with several experiments illustrating the effectiveness of our approach in practice.

show abstract

On the Sample Complexity of the Linear Quadratic Regulator

Cited by 82 publications

References 36 publications

Linear Systems can be Hard to Learn

Linear Systems can be Hard to Learn

System Level Synthesis

Task-Optimal Exploration in Linear Dynamical Systems

Contact Info

Product

Resources

About