Reinforcement learning and optimal adaptive control: An overview and implementation examples

Khan, Said Ghani; Herrmann, Guido; Lewis, Frank L.; Pipe, Anthony G.; Melhuish, Chris

doi:10.1016/j.arcontrol.2012.03.004

Cited by 182 publications

(69 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The ADP technology is used widely in aircraft [25], robot arm [33], Micro-electromechanical-system actuator [34] and turbocharged diesel engine [21] and so on. At present, we just study our method in theory.…”

Section: Resultsmentioning

confidence: 99%

Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator

Zhang

Liu

et al. 2015

ISA Transactions

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator

Zhang

Liu

et al. 2015

ISA Transactions

View full text Add to dashboard Cite

“…The stability analysis is barely objective but the optimum controller is usually preferred [6] for several practical control systems. By solving the nonlinear HamiltoneJacobieBellman (HJB), adaptive dynamic programming (ADP) schemes had been developed to minimize an infinite cost function for both the error signal and the control effort [7].…”

Section: Introductionmentioning

confidence: 99%

A data-driven adaptive controller for a class of unknown nonlinear discrete-time systems with estimated PPD

Treesatayapun

2015

Engineering Science and Technology, an International Journal

View full text Add to dashboard Cite

a b s t r a c tAn adaptive control scheme based on data-driven controller (DDC) is proposed in this article. Unlike several DDC techniques, the proposed controller is constructed by an adaptive fuzzy rule emulated network (FREN) which is able to include human knowledge based on controlled plant's inputeoutput signals within the format of IF-THEN rules. Regarding to this advantage, an on-line estimation of pseudo partial derivative (PPD) and resetting algorithms, which are commonly used by DDC, can be omitted here. Furthermore, a novel adaptive algorithm is introduced to minimize for both tracking error and control effort with stability analysis for the closed-loop system. The experimental system with brushed DC-motor current control is constructed to validate the performance of the proposed control scheme. Comparative results with conventional DDC and radial basis function (RBF) controllers demonstrate that the proposed controller can provide the less tracking error and minimize the control effort.

show abstract

“…As a major control technique of handling parametric uncertainties, adaptive control still remains among the most active research fields in the control community. [1][2][3][4][5][6][7] Usually, adaptive control only achieves asymptotic convergence of tracking errors and does not guarantee convergence of parameter estimation errors without a condition termed persistent excitation (PE). 8 Parameter convergence in adaptive control is desirable as it enhances the overall stability and robustness properties of the closed-loop system, 9 where the robustness properties result from closed-loop exponential stability, and detailed analysis can be referred to the work of Sastry and Bodson.…”

Section: Introductionmentioning

confidence: 99%

Efficient learning from adaptive control under sufficient excitation

Pan

Aranovskiy

Bobtsov

et al. 2019

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Summary Parameter convergence is desirable in adaptive control as it enhances the overall stability and robustness properties of the closed‐loop system. In existing online historical data (OHD)–driven parameter learning schemes, all OHD are exploited to update parameter estimates such that parameter convergence is guaranteed under a sufficient excitation (SE) condition which is strictly weaker than the classical persistent excitation condition. Nevertheless, the exploitation of all OHD not only results in possible unbounded adaptation but also loses the flexibility of handling slowly time‐varying uncertainties. This paper presents an efficient OHD‐driven parameter learning scheme for adaptive control, where a variable forgetting factor is specifically designed and is equipped with an estimation error feedback such that exponential parameter convergence is achieved under the SE condition without the aforesaid drawbacks. The proposed parameter learning scheme is incorporated with direct adaptive control to construct an OHD‐based composite learning control strategy. Numerical results have verified the effectiveness of the proposed approach.

show abstract

Reinforcement learning and optimal adaptive control: An overview and implementation examples

Cited by 182 publications

References 57 publications

Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator

Model-free optimal controller design for continuous-time nonlinear systems by adaptive dynamic programming based on a precompensator

A data-driven adaptive controller for a class of unknown nonlinear discrete-time systems with estimated PPD

Efficient learning from adaptive control under sufficient excitation

Contact Info

Product

Resources

About