Approximate dynamic programming for continuous-time linear quadratic regulator problems: relaxation of known input-coupling matrix assumption

Lee, J. Y.; Park, J. B.; Choi, Yoon Ho

doi:10.1049/iet-cta.2010.0521

Cited by 13 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process corresponds to the idea of off‐policy in the RL literatures, which refers to the fact that the executed policies are different to the estimated ones. Such mechanism is extended to non‐linear systems in [28, 29]. Another intelligent technique in the field of RL, experience replay (ER), is successfully applied by Modares et al [30] in their adaptive algorithm.…”

Section: Introductionmentioning

confidence: 99%

Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics

Zhu

Zhao

2016

IET Control Theory & Applications

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics

Zhu

Zhao

2016

IET Control Theory & Applications

View full text Add to dashboard Cite

“…However, for some complicated nonlinear systems, the method of choosing the internal parameters is still an intractable problem. One available approach is given in the work of Lee et al (2012), which is only discussed for linear systems.…”

Section: The Critic Network Design and Stability Analysismentioning

confidence: 99%

Compensator-based approximate optimal control for affine nonlinear systems with input constraints and unmatched disturbances

Lü

Liu

Sun

et al. 2020

Transactions of the Institute of Measurement and Control

View full text Add to dashboard Cite

This paper develops a novel approximate optimal control method for a class of constrained continuous-time nonlinear systems in the presence of disturbances via adaptive dynamic programming (ADP) technique. First, an auxiliary dynamic compensator is introduced to deal with the input constraints. Through augmenting the original system with the designed auxiliary compensator, the design of constrained optimal controller is circumvented by stabilizing the equivalent augmented system. Then, the cost function is appropriately redefined by introducing an additional function connected with disturbances for the augmented nominal system in order to compensate the effect of unmatched disturbances. Next, the solution of associated Hamilton-Jacobi-Bellman (HJB) equation is solved online with weight adaptation law using neural networks (NNs). Furthermore, an additional robustifying term is utilized to compensate the effect of the approximation error of NNs, and thus the asymptotic stability of the closed-loop system is guaranteed. Finally, all signals of the closed-loop system are proved to be asymptotic convergence by using Lyapunov method. Simulation examples demonstrate the effectiveness of the proposed scheme.

show abstract

“…The outer iteration aims to achieve the optimal Q-function. However, the iterative control sequence cannot be obtained directly by solving (29) and (31). Therefore, the inner iteration is necessarily required.…”

Section: Algorithm Derivationmentioning

confidence: 99%

“…Adaptive dynamic programming (ADP), proposed by Werbos [21,22], has demonstrated powerful self-learning capability for optimisation of complex non-linear systems [23][24][25][26][27][28][29][30][31]. Q-learning is a typical method of ADP proposed by Watkins [32,33].…”

Section: Introductionmentioning

confidence: 99%

Echo state network‐based Q‐learning method for optimal battery control of offices combined with renewable energy

Shi

Liu

Wei

2017

IET Control Theory & Applications

View full text Add to dashboard Cite

An echo state network (ESN)-based Q-learning method is developed for optimal energy management of an office, where the solar energy is introduced as the renewable source, and a battery is installed with a control unit. The energy consumption in the office, also considered as the energy demand, is separated into those from sockets, lights and airconditioners. First, ESNs, well known for their excellent modelling performance for time series, are employed to model the time series of the real-time electricity rate, renewable energy and energy demand as periodic functions. Second, given the periodic models of the electricity rate, renewable energy and energy demand, an ESN-based Q-learning method with the Q-function approximated by an ESN is developed and implemented to determine the optimal charging/discharging/idle strategies for the battery in the office, so that the total cost of electricity from the grid can be reduced. Finally, numerical analysis is conducted to illustrate the performance of the developed method.

show abstract

Approximate dynamic programming for continuous-time linear quadratic regulator problems: relaxation of known input-coupling matrix assumption

Cited by 13 publications

References 24 publications

Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics

Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics

Compensator-based approximate optimal control for affine nonlinear systems with input constraints and unmatched disturbances

Echo state network‐based Q‐learning method for optimal battery control of offices combined with renewable energy

Contact Info

Product

Resources

About