Near-Optimal Controller for Nonlinear Continuous-Time Systems With Unknown Dynamics Using Policy Iteration

Abstract

This paper presents a single-network adaptive critic-based controller for continuous-time systems with unknown dynamics in a policy iteration (PI) framework. It is assumed that the unknown dynamics can be estimated using the Takagi-Sugeno-Kang fuzzy model with arbitrary precision. The successful implementation of a PI scheme depends on the effective learning of critic network parameters. Network parameters must stabilize the system in each iteration in addition to approximating the critic and the cost. It is found that the critic updates according to the Hamilton-Jacobi-Bellman formulation sometimes lead to the instability of the closed-loop systems. In the proposed work, a novel critic network parameter update scheme is adopted, which not only approximates the critic at current iteration but also provides feasible solutions that keep the policy stable in the next step of training by combining a Lyapunov-based linear matrix inequalities approach with PI. The critic modeling technique presented here is the first of its kind to address this issue. Though multiple literature exists discussing the convergence of PI, however, to the best of our knowledge, there exists no literature, which focuses on the effect of critic network parameters on the convergence. Computational complexity in the proposed algorithm is reduced to the order of (Fz)(n-1) , where n is the fuzzy state dimensionality and Fz is the number of fuzzy zones in the states space. A genetic algorithm toolbox of MATLAB is used for searching stable parameters while minimizing the training error. The proposed algorithm also provides a way to solve for the initial stable control policy in the PI scheme. The algorithm is validated through real-time experiment on a commercial robotic manipulator. Results show that the algorithm successfully finds stable critic network parameters in real time for a highly nonlinear system.

Back-Stepping Fuzzy Adaptive Sliding Mode Trajectory Tracking Control for Wall-Climbing Robot

“…In this section, we verify the effectiveness and stability of the proposed control law through experiment. The mathematical model of a four-wheel drive wall-climbing robot is described by equation (10). The main parameters of the four-wheel-driven wall-climbing robot：M =4.8 kg, J = 3kg.m 2 xy ) converges to zero.…”

“…The kinematics model is basically the same as the ground-wheel-climbing robot [9]. It is important that the dynamic model of the wheeled wall-climbing robot allows for the influence of adsorption force and gravity [10]. Therefore, in order to solve the problem of trajectory tracking in the wheeled wall-climbing robot, it is necessary to consider the influence of the force and instability factors to establish a complete dynamic model [11]- [13].…”

Neural Adaptive Backstepping Control of a Robotic Manipulator With Prescribed Performance Constraint

“…Yang and Liu [21] adopted adaptive RBFNN control to improve the position tracking performance of the coupled motor drives system. Dutta et al [22] presented a single-network adaptive critic-based controller for continuoustime systems with unknown dynamics in a policy iteration framework. This control algorithm is verified in a commercial robotic manipulator experiment.…”