Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System

Boczar, Ross; Matni, Nikolai; Recht, Benjamin

doi:10.48550/arxiv.1803.09186

Cited by 9 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lacking in the output feedback setting however is an analogous family of robustness results to those presented in Section 4.5. Although preliminary results exist [22], this remains in important direction for future work.…”

Section: Discussionmentioning

confidence: 95%

“…In particular, we know of no results in the literature that bound the degradation in performance of controlling an uncertain system in terms of the size of the perturbations affecting it. While traditional results in robust control considered fixed model uncertainty, in the data-driven and learning based control setting, analytic interpretability of the effects uncertainty size are essential in obtaining sample-complexity bounds for these methods [19,20,21,22].…”

Section: Robust Controlmentioning

confidence: 99%

See 1 more Smart Citation

System Level Synthesis

Anderson¹,

Doyle²,

Low³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

This article surveys the System Level Synthesis framework, which presents a novel perspective on constrained robust and optimal controller synthesis for linear systems. We show how SLS shifts the controller synthesis task from the design of a controller to the design of the entire closed loop system, and highlight the benefits of this approach in terms of scalability and transparency. We emphasize two particular applications of SLS, namely large-scale distributed optimal control and robust control. In the case of distributed control, we show how SLS allows for localized controllers to be computed, extending robust and optimal control methods to largescale systems under practical and realistic assumptions. In the case of robust control, we show how SLS allows for novel design methodologies that, for the first time, quantify the degradation in performance of a robust controller due to model uncertainty -such transparency is key in allowing robust control methods to interact, in a principled way, with modern techniques from machine learning and statistical inference. Throughout, we emphasize practical and efficient computational solutions, and demonstrate our methods on easy to understand case studies.

show abstract

Section: Discussionmentioning

confidence: 95%

Section: Robust Controlmentioning

confidence: 99%

System Level Synthesis

Anderson¹,

Doyle²,

Low³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, estimated models (Levine & Koltun, 2013;Gu et al, 2016;Kalweit & Boedecker, 2017) On the control theory side, Dean et al (2018; provide strong finite sample complexity bounds for solving linear quadratic regulator using model-based approach. Boczar et al (2018) provide finitedata guarantees for the "coarse-ID control" pipeline, which is composed of a system identification step followed by a robust controller synthesis procedure. Our method is inspired by the general idea of maximizing a low bound of the reward in (Dean et al, 2017).…”

Section: Additional Related Workmentioning

confidence: 99%

Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees

Luo,

Xu,

et al. 2018

Preprint

View full text Add to dashboard Cite

Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding of such methods has been rather limited. This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algorithms with theoretical guarantees. We design a meta-algorithm with a theoretical guarantee of monotone improvement to a local maximum of the expected reward. The meta-algorithm iteratively builds a lower bound of the expected reward based on the estimated dynamical model and sample trajectories, and then maximizes the lower bound jointly over the policy and the model. The framework extends the optimism-in-face-of-uncertainty principle to non-linear dynamical models in a way that requires no explicit uncertainty quantification. Instantiating our framework with simplification gives a variant of model-based RL algorithms Stochastic Lower Bounds Optimization (SLBO). Experiments demonstrate that SLBO achieves stateof-the-art performance when only one million or fewer samples are permitted on a range of continuous control benchmark tasks. 1

show abstract

“…Several other authors [28,17,30] have derived similarly important results at the same time. Subsequently, a number of authors [11,1,14,15,3,7] have applied them to the (Linear-Quadratic, LQ) control of an unknown system, which underlies much of reinforcement learning.…”

Section: Related Workmentioning

confidence: 99%

“…In improper learning of such an LDS (which we refer to as an identification problem), one wishes to estimate ŷk such that ŷk are close to the best estimates y * k of y k possible at time k. When there is no hidden state, the identification problem is convex and a variety of methods work well. When there is a hidden state, the problem is non-convex and only rather recently spectral filtering [19,18] has been used to obtain identification procedures with regret bounded by Õ(log 7 √ k) at time k, where Õ(•) hides terms that depend polynomially on the dimension of the system and norms of the inputs and outputs and the noise.…”

Section: Introductionmentioning

confidence: 99%

Robust Spectral Filtering and Anomaly Detection

Marecek,

Tchrakian

2018

Preprint

View full text Add to dashboard Cite

We consider a setting, where the output of a linear dynamical system (LDS) is, with an unknown but fixed probability, replaced by noise. There, we present a robust method for the prediction of the outputs of the LDS and identification of the samples of noise, and prove guarantees on its statistical performance. One application lies in anomaly detection: the samples of noise, unlikely to have been generated by our estimate of the unknown dynamics, can be flagged to operators of the system for further study.

show abstract

Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System

Cited by 9 publications

References 0 publications

System Level Synthesis

System Level Synthesis

Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees

Robust Spectral Filtering and Anomaly Detection

Contact Info

Product

Resources

About