The residual gradient FACL algorithm for differential games

Abstract

A new fuzzy reinforcement learning algorithm that tunes the input and the output parameters of a fuzzy logic controller is proposed in this paper. The proposed algorithm uses three fuzzy inference systems (FISs); one is used as an actor (fuzzy logic controller, FLC), and the other two FISs are used as critics. The proposed algorithm uses the residual gradient value iteration algorithm described in [4] to tune the input and the output parameters of the actor (FLC) of the learning robot. The proposed algorithm also tunes the input and the output parameters of the critics. The proposed algorithm is called the residual gradient fuzzy actor critics learning (RGFACL) algorithm. The proposed algorithm is used to learn a single pursuit-evasion differential game. Simulation results show that the performance of the proposed RGFACL algorithm outperforms the performance of the fuzzy actor critic learning (FACL) and the Q-learning fuzzy inference system (QLFIS) algorithms proposed in [3] and [7], respectively, in terms of convergence and speed of learning.

Decentralized learning in pursuit-evasion differential games with multi-pursuer and single-superior evader

“…Different fuzzy learning algorithms that can be applied to pursuit-evasion differential games are proposed in the literature [2]- [4]. In this work, the proposed algorithm uses the RGFACL algorithm as the RGFACL algorithm is shown in [2] to be robust and has a quick convergence speed.…”

“…In this work, the proposed algorithm uses the RGFACL algorithm as the RGFACL algorithm is shown in [2] to be robust and has a quick convergence speed. The RGFACL algorithm uses three fuzzy inference systems (FISs); one is used as an actor (fuzzy logic controller, FLC), and the other two FISs are used as critics [2]. The critics are used to estimate the value functions V t (s t ) and V t (s t+1 ) of the same learning agent at two different states s t and s t+1 , respectively.…”

“…For more information on how to calculate the derivative ∂ut ∂ψ A t , the reader may refer to the work presented in [2].…”

“…Our objective in this work is to make the pursuers in a multi-pursuer singlesuperior-evader pursuit-evasion differential game learn their strategies so that one or some of the learning pursuers can capture the superior evader. The proposed algorithm uses the RGFACL algorithm proposed in [2] to tune the parameters of the FLC of each learning pursuer P i . The proposed algorithm is a decentralized algorithm as each learning pursuer P i has its own learning algorithm and no communication among pursuers is required.…”

“…Pursuit-evasion games have been widely studied in the literature [2], [3], [5]- [16]. Pursuit-evasion games have extensive applications in the real world such as surveillance and tracking, search and rescue, locating and capturing hostile intruders, localizing and neutralizing environmental threats, and collision avoiding systems in intelligent transportation systems [1], [6], [8], [12], [13].…”