Approximate Dynamic Programming Strategy for Dual Adaptive Control

Abstract

An approximate dynamic programming (ADP) strategy for a dual adaptive control problem is presented. An optimal control policy of a dual adaptive control problem can be derived by solving a stochastic dynamic programming problem, which is computationally intractable using conventional solution methods that involve sampling of a complete hyperstate space. To solve the problem in a computationally amenable manner, we perform closed-loop simulations with different control policies to generate a data set that defines a subset of a hyperstate within which the Bellman equation is iterated. A local approximator with a penalty function is designed for estimation of cost-to-go values over the continuous hyperstate space. An integrating process with an unknown gain is used for illustration.

Simulation‐Based Design of Dual‐Mode Controller for Non‐Linear Processes

“…A caveat in the above procedure is that the cost-to-go estimate can leave signifi cant bias in regions of the state space can be addressed by adopting proper data fi ltering techniques (Hastie et al, 2001). In addition, the ADP scheme coupled with an observer has been shown to be applicable to uncertain systems with parametric uncertainties (Lee and Lee, 2005b), and it could be modifi ed to learn a cost-to-go function even without a process model (Lee and Lee, 2005a).…”