Neural networks for control and system identification

Werbos, Paul J.

doi:10.1109/cdc.1989.70114

Cited by 261 publications

(151 citation statements)

References 5 publications

Supporting

Mentioning

150

Contrasting

Unclassified

Order By: Relevance

“…The conditions (13), (14), and (15) are used to propose the RL solution framework as will be explained in the next section.…”

Section: B Hamiltonian Dynamicsmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning Solution with Costate Approximation for a Flexible Wing Aircraft

Abouheaf

Gueaieb

2018

2018 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applicati

View full text Add to dashboard Cite

Abstract-An online adaptive learning approach based on costate function approximation is developed to solve an optimal control problem in real time. The proposed approach tackles the main concerns associated with the classical Dual Heuristic Dynamic Programming techniques in uncertain dynamical environments. It employs a policy iteration paradigm along with adaptive critics to implement the adaptive learning solution. The resultant framework does not need or require prior knowledge of the system dynamics, which makes it suitable for systems with high modeling uncertainties. As a proof of concept, the suggested structure is applied for the auto-pilot control of a flexible wing aircraft with unknown dynamics which are continuously varying at each trim speed condition. Numerical simulations showed that the adaptive control technique was able to learn the system's dynamics and regulate its states as desired in a relatively short time.

show abstract

“…The conditions (13), (14), and (15) are used to propose the RL solution framework as will be explained in the next section.…”

Section: B Hamiltonian Dynamicsmentioning

confidence: 99%

“…Following this assessment, the actor and critic weights are updated [12], [13]. The adaptive critics are used to solve the optimal control problem in realtime in [14]. The optimal control problem finds the necessity optimality conditions and hence the optimal strategies [15].…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning Solution with Costate Approximation for a Flexible Wing Aircraft

Abouheaf

Gueaieb

2018

2018 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applicati

View full text Add to dashboard Cite

show abstract

“…This allows approximate solution of (10), (11). In this section, we review how to implement the HDP value iterations algorithm with two parametric structures such as neural networks (Werbos, 1990) and (Lewis & Jaganathan, 1999). The important point is stressed that the use of two NN, a critic for value function approximation and an action NN for the control, allows the implementation of HDP in the LQR case without knowing the system internal dynamics matrix A.…”

Section: Neural Network Approximation For Value and Actionmentioning

confidence: 99%

“…The HDP value iteration algorithm (Werbos, 1990) is a method to solve the DT HJB online. In this section, a proof of convergence of the HDP algorithm in the general nonlinear discrete-time setting is presented.…”

Section: The Hdp Algorithmmentioning

confidence: 99%

Heuristic Dynamic Programming Nonlinear Optimal Controller

Al-Tamimi¹,

Abu-Khalaf²,

Lewis³

2010

Application of Machine Learning

View full text Add to dashboard Cite

“…The first RL applications on control systems have been found on Werbos [19,20], where the regulation problem was tackled, whose objective is to design a controller for a given process, such that the internal state of this process approaches zero as time increases unbounded. Then, an immediate extension was to apply policy iteration (PI) algorithms to solve the linear quadratic regulator (LQR) problem [4].…”

Section: Introductionmentioning

confidence: 99%

Stabilizing Dynamic State Feedback Controller Synthesis: A Reinforcement Learning Approach

Solís

Olivares

Allende

2016

SIC

View full text Add to dashboard Cite

State feedback controllers are appealing due to their structural simplicity. Nevertheless, when stabilizing a given plant, dynamics of this type of controllers could lead the static feedback gain to take higher values than desired. On the other hand, a dynamic state feedback controller is capable of achieving the same or even better performance by introducing additional parameters into the model to be designed. In this document, the Linear Quadratic Tracking problem will be tackled using a (linear) dynamic state feedback controller, whose parameters will be chosen by means of applying reinforcement learning techniques, which have been proved to be especially useful when the model of the plant to be controlled is unknown or inaccurate.

show abstract

Neural networks for control and system identification

Cited by 261 publications

References 5 publications

Reinforcement Learning Solution with Costate Approximation for a Flexible Wing Aircraft

Reinforcement Learning Solution with Costate Approximation for a Flexible Wing Aircraft

Heuristic Dynamic Programming Nonlinear Optimal Controller

Stabilizing Dynamic State Feedback Controller Synthesis: A Reinforcement Learning Approach

Contact Info

Product

Resources

About