Adaptive Control with Approximated Policy Search Approach

Abstract

Most of existing adaptive control schemes are designed to minimize error between plant state and goal state despite the fact that executing actions that are predicted to result in smaller errors only can mislead to non-goal states. We develop an adaptive control scheme that involves manipulating a controller of a general type to improve its performance as measured by an evaluation function. The developed method is closely related to a theory of Reinforcement Learning (RL) but imposes a practical assumption made for faster learning. We assume that a value function of RL can be approximated by a function of Euclidean distance from a goal state and an action executed at the state. And, we propose to use it for the gradient search as an evaluation function. Simulation results provided through application of the proposed scheme to a pole-balancing problem using a linear state feedback controller and fuzzy controller verify the scheme’s efficacy

Evaluation-Function-based Model-free Adaptive Fuzzy Control

“…AEÇo = K È É sin 3 0O P . For comparison, the application of AFC to both control problems were simulated using the following three cases: Note that cases 1 and 2 are the same as considered in [18], in which Eq. (21) was originally proposed, the update rule with failure detector.…”

“…They are not applicable to plants that cannot be represented by the assumed model structure. Previous research on the same issue as addressed in this paper is reported in [15][16][17][18]. Although the proposed systems work, their performance is poor.…”

“…In other words, smaller errors do not necessarily indicate better control performance. Some methods for solving control problems based on the RL approach have been proposed [18,20,22,23]. They need real-time learning of the value function.…”