Learning Robust LQ-Controllers Using Application Oriented Exploration

Ferizbegovic, Mina; Umenberger, Jack; Hjalmarsson, Håkan; Schön, Thomas B.

doi:10.1109/lcsys.2019.2921512

Cited by 32 publications

(64 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This bound, derived for the static case in ( [12], Lemma 1), is tight whenK l = K l . Here it will be usedK l = K DRDE l .…”

Section: Ktmentioning

confidence: 84%

“…It is noted that π 2 (K 2 t , 0) does not represent the robust policy redesigned with the new uncertain model, as this would be computed a-posteriori by solving problem (8b) with the new uncertainty set, and is only a fictitious policy introduced to allow the definition of the worst-case cost J π2 2 as a functional of the uncertainty. While inspired by their application-oriented strategy, the problem in ( 10) is distinct from those solved in [12], [13]. Differences, which force the problem to be solved with a different approach, include: the finite horizon setting (also used in [13]); the multiobjective formulation; and the use of the expected system response to update the uncertainty.…”

Section: Problem Statementmentioning

confidence: 99%

“…using regrets to prescribe a rate of decay of random actions), and do not directly leverage the properties of the system. The concept of structured exploration, inspired by ideas initially presented in [12], was recently proposed in [13] to address some of these aspects. The problem is framed within the dual Linear Quadratic Regulator (LQR) setting, commonly used as a baseline for data-driven algorithms [14], [15], [10], [16], [11].…”

mentioning

confidence: 99%

“…However, an important feature in [12], [13] is that the worst-case system's response is used to update the ellipsoidal uncertainty set. This means that, in practice, the uncertainty reduction for which the dual policy is optimized can only be achieved if the true system coincides with the worst-case one in the ellipsoidal set.…”

mentioning

confidence: 99%

See 3 more Smart Citations

A Multiobjective LQR Synthesis Approach to Dual Control for Uncertain Plants

Iannelli

Smith

2020

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

The paper proposes a dual control finite horizon LQR synthesis procedure for unknown systems characterized by mean and covariance estimates. The optimized policy comprises time-varying statefeedback and dithering components, and the control problem is framed as a multiobjective synthesis which seeks a balance between exploitation and exploration costs. It is shown that classic experiment design problems can be recast in this framework by replacing the exploitation cost with an information reward. Numerical examples demonstrate the different dual control trade-offs on plants with different properties.

show abstract

“…This bound, derived for the static case in ( [12], Lemma 1), is tight whenK l = K l . Here it will be usedK l = K DRDE l .…”

Section: Ktmentioning

confidence: 84%

Section: Problem Statementmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

A Multiobjective LQR Synthesis Approach to Dual Control for Uncertain Plants

Iannelli

Smith

2020

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

show abstract

“…Using the vertex representation (16), the condition that (2) is satisfied for all x ∈  k is equivalent to, for all j ∈ N [1,m] ,…”

Section: Polytopic Tubes For Robust Constraint Satisfactionmentioning

confidence: 99%

Robust adaptive model predictive control: Performance and parameter estimation

Liu

Cannon

Koksal-Rivet

2020

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

For systems with uncertain linear models, bounded additive disturbances and state and control constraints, a robust model predictive control (MPC) algorithm incorporating online model adaptation is proposed. Sets of model parameters are identified online and employed in a robust tube MPC strategy with a nominal cost. The algorithm is shown to be recursively feasible and input-to-state stable.Computational tractability is ensured by using polytopic sets of fixed complexity to bound parameter sets and predicted states. Convex conditions for persistence of excitation are derived and are related to probabilistic rates of convergence and asymptotic bounds on parameter set estimates. We discuss how to balance conflicting requirements on control signals for achieving good tracking performance and parameter set estimate accuracy. Conditions for convergence of the estimated parameter set are discussed for the case of fixed complexity parameter set estimates, inexact disturbance bounds, and noisy measurements.

show abstract

The role of identification in data‐driven policy iteration: A system theoretic study

Song,

Iannelli

2024

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

The goal of this article is to study fundamental mechanisms behind so‐called indirect and direct data‐driven control for unknown systems. Specifically, we consider policy iteration applied to the linear quadratic regulator problem. Two iterative procedures, where data collected from the system are repeatedly used to compute new estimates of the desired optimal controller, are considered. In indirect policy iteration, data are used to obtain an updated model estimate through a recursive identification scheme, which is used in a certainty‐equivalent fashion to perform the classic policy iteration update. By casting the concurrent model identification and control design as a feedback interconnection between two algorithmic systems, we provide a closed‐loop analysis that shows convergence and robustness properties for arbitrary levels of excitation in the data. In direct policy iteration, data are used to approximate the value function and design the associated controller without requiring the intermediate identification step. After proposing an extension to a recently proposed scheme that overcomes potential identifiability issues, we establish under which conditions this procedure is guaranteed to deliver the optimal controller. Based on these analyses we are able to compare the strengths and limitations of the two approaches, highlighting aspects such as the required samples, convergence properties, and excitation requirement. Simulations are also provided to illustrate the results.

show abstract

Learning Robust LQ-Controllers Using Application Oriented Exploration

Cited by 32 publications

References 22 publications

A Multiobjective LQR Synthesis Approach to Dual Control for Uncertain Plants

A Multiobjective LQR Synthesis Approach to Dual Control for Uncertain Plants

Robust adaptive model predictive control: Performance and parameter estimation

The role of identification in data‐driven policy iteration: A system theoretic study

Contact Info

Product

Resources

About