Recursive Least-squares Reinforcement Learning Controller Based on General Fuzzy CMAC

Abstract

Combined CMAC addressing schemes with fuzzy logic idea, a general fuzzy CMAC (GFAC) is proposed, in which the fuzzy membership functions are utilized as the receptive field functions. The mapping of receptive field functions, the selection law of membership function and the learning algorithm are presented. Recursive least-squares temporal difference algorithm (RLS-TD) is deduced, which can use data more efficiently with fast convergence and less computational burden. Using RLS-TD method a reinforcement learning structure based on GFAC is applied to ship steering control, as provides an efficient way for the improvement of ship steering control performance. The parameters of controller are online learned and adjusted. Simulation results show that the ship course can be properly controlled in case of the disturbances of wave and wind. It is demonstrated that the proposed algorithm is a promising alternative to conventional autopilots.