A Neural Network PID-Like Controller Using a Hybrid of Online Actor-Critic Reinforcement Algorithm with the Square Root Cubature Kalman Filter

Abstract

This paper presents a new model of the Neural Network PID-Like controller using an Actor-Critic reinforcement algorithm, called the Neural Network PID-Like controller using an Actor-Critic reinforcement algorithm (NNPID-AC). The proposed NNPID-AC controller is designed to develop the performances and the speed of calculation under the iterative learning algorithm. In the learning algorithm, the critic algorithm receives the reward value and control input to criticize the current state using the action-state value function approximation. Furthermore, instead of applying every available action to predict the local successor state, the algorithm only uses one-step estimation using the fifth degree spherical-radial cubature rule algorithm. To evaluate the proposed NNPID-AC controller, the robot arm MATLAB simulations have been implemented and provide the control system with the load and noise to prove the robustness and fault tolerance, respectively. From the results, the robot arm control system simulation under the control of the proposed NNPID-AC controller can potentially track the error and gives the best responses compared with the other conventional controller either with or without the load and the noise disturbance.