Event-Triggered Learning for Linear Quadratic Control

Schlüter, Henning; Solowjow, Friedrich; Trimpe, Sebastian

doi:10.1109/tac.2020.3030877

Cited by 13 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The general idea of ETL, i.e., learning only when necessary, has recently been proposed in [1]- [8], but in different settings than the one herein. Specifically, prior work on ETL builds triggers for a specific task or application.…”

Section: A Related Workmentioning

confidence: 99%

“…In particular, [2], [3] provide the theory to reduce communication in networked control systems and was experimentally validated with IMUsensor networks that monitor human gaits [4], [5]. In [6], ETL was extended to event-triggered pulse control and in [1] to LQR control. All of the above papers are not directly considering the learning problem and, further, do not leverage the statistical information that the learning trigger provides.…”

Section: A Related Workmentioning

confidence: 99%

“…Next, we show the proposed ETL framework in a numerical simulation 1 . In particular, we demonstrate the following properties:…”

Section: Simulation Examplementioning

confidence: 99%

“…There is prior work on ETL that is closely tailored to specific applications and downstream tasks, such as linear quadratic control [1] or the reduction of communication in networked control systems [2]. The triggers developed therein are based on a signal that is specific to the task: quadratic control performance [1] and communication rates [2]. While this is helpful for the specific task, it also limits the algorithm to exactly that application.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Parameter Filter-based Event-triggered Learning

Schlor¹,

Solowjow²,

Trimpe³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Model-based algorithms are deeply rooted in modern control and systems theory. However, they usually come with a critical assumption -access to an accurate model of the system. In practice, models are far from perfect. Even precisely tuned estimates of unknown parameters will deteriorate over time. Therefore, it is essential to detect the change to avoid suboptimal or even dangerous behavior of a control system. We propose to combine statistical tests with dedicated parameter filters that track unknown system parameters from state data. These filters yield point estimates of the unknown parameters and, further, an inherent notion of uncertainty. When the point estimate leaves the confidence region, we trigger active learning experiments. We update models only after enforcing a sufficiently small uncertainty in the filter. Thus, models are only updated when necessary and statistically significant while ensuring guaranteed improvement, which we call event-triggered learning. We validate the proposed method in numerical simulations of a DC motor in combination with model predictive control.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

“…Next, we show the proposed ETL framework in a numerical simulation 1 . In particular, we demonstrate the following properties:…”

Section: Simulation Examplementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Parameter Filter-based Event-triggered Learning

Schlor¹,

Solowjow²,

Trimpe³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is worth mentioning that following this line, numerous researchers begin to revisit the linear-quadratic regulator (LQR) problem [14][15][16][17][18][19], the task of which is to determine an optimal feedback controller minimizing a quadratic cost function for linear systems [20]. One important reason for the revisit of the LQR problem is that it is simple and yet captures the two characteristics of RL problems mentioned in the previous paragraph.…”

Section: Introductionmentioning

confidence: 99%

Model-Free Design of Stochastic LQR Controller from Reinforcement Learning and Primal-Dual Optimization Perspective

Li,

Qin,

Zheng

et al. 2021

Preprint

View full text Add to dashboard Cite

To further understand the underlying mechanism of various reinforcement learning (RL) algorithms and also to better use the optimization theory to make further progress in RL, many researchers begin to revisit the linear-quadratic regulator (LQR) problem, whose setting is simple and yet captures the characteristics of RL. Inspired by this, this work is concerned with the model-free design of stochastic LQR controller for linear systems subject to Gaussian noises, from the perspective of both RL and primal-dual optimization. From the RL perspective, we first develop a new model-free off-policy policy iteration (MF-OPPI) algorithm, in which the sampled data is repeatedly used for updating the policy to alleviate the data-hungry problem to some extent. We then provide a rigorous analysis for algorithm convergence by showing that the involved iterations are equivalent to the iterations in the classical policy iteration (PI) algorithm. From the perspective of optimization, we first reformulate the stochastic LQR problem at hand as a constrained non-convex optimization problem, which is shown to have strong duality. Then, to solve this non-convex optimization problem, we propose a model-based primal-dual (MB-PD) algorithm based on the properties of the resulting Karush-Kuhn-Tucker (KKT) conditions. We also give a model-free implementation for the MB-PD algorithm by solving a transformed dual feasibility condition. More importantly, we show that the dual and primal update steps in the MB-PD algorithm can be interpreted as the policy evaluation and policy improvement steps in the PI algorithm, respectively. Finally, we provide one simulation example to show the performance of the proposed algorithms.

show abstract

Model‐free self‐triggered control based on deep reinforcement learning for unknown nonlinear systems

Wan

Karimi

Luan

et al. 2022

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

This article proposes a joint learning technique for control inputs and triggering intervals of self‐triggered control nonlinear systems with unknown dynamics. First, deep reinforcement learning is introduced to the self‐triggered control system by considering both the control performance and triggering performance in the reward function. Then, the control inputs and triggering intervals are simultaneously learned by the developed deep deterministic policy gradient approach. Under this strategy, not only the desired control performance is guaranteed for unknown nonlinear systems, but also both the computation and communication occupation for the controlled system are decreased without any triggering thresholds. Finally, simulations for the cart‐pole swing‐up system are illustrated to verify the effectiveness of the proposed scheme.

show abstract

Event-Triggered Learning for Linear Quadratic Control

Cited by 13 publications

References 34 publications

Parameter Filter-based Event-triggered Learning

Parameter Filter-based Event-triggered Learning

Model-Free Design of Stochastic LQR Controller from Reinforcement Learning and Primal-Dual Optimization Perspective

Model‐free self‐triggered control based on deep reinforcement learning for unknown nonlinear systems

Contact Info

Product

Resources

About