Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle

Abstract

Selected Adaptive Critic (AC) methods are known to be capable of designing (approximately) optimal control policies for non-linear plants (in the sense of approximating Bellman Dynamic Programming). The present research focuses on an AC method known as Dual Heuristic Programming. There are many issues related to the pragmatics of successfully applying the AC methods, but now that the operational aspects of the DHP method are becoming refined and better understood, it is instructive to carry out empirical research with the method, to inform theoretical research being carried out in parallel. In particular; it is seen as useful to explore correspondences between the form of a Utility function and the resulting controllers designed by the DHP method. The task of designing a steering controller for a 2-axle, terrestrial, autonomous vehicle is the basis of the empirical work reported here (and in a companion paper). The new aspect in the present paper relates to using a pair of critics (distinct from the shadow critics described elsewhere by the authors) to "divide the labor" of training the controller. Improvements in convergence of the training process is realized in this way. The controllers designed by the DHP method are pleasingly robust, and demonstrate good performance on disturbances not even trained on --1.) encountering a patch of ice during a steering maneuver; and 2.) encountering a wind gust perpendicular to direction of travel.

Online Adaptive Critic Flight Control

“…Adaptive critic controllers have been successfully trained offline for two-axle vehicle steering and speed control, 5 agile missile interception, 6 aircraft autolanding and control, 7−9 and turbogenerator control. 4 Although several architectures have been proposed to accelerate convergence to the optimal solution (as in Refs.…”

Vector control of a grid-connected rectifier/inverter using an artificial neural network

“…Adaptive critic designs constitute a class of approximate dynamic programming (ADP) methods that use incremental optimization combined with parametric structures that approximate the optimal cost and the control [22,23]. Both classical DP and ADP methods have been used to train neural networks for a large number of nonlinear control applications, such as steering and controlling the speed of a two-axle vehicle [24], intercepting an agile missile [25], performing auto landing and control of an aircraft [26][27][28], and controlling a turbogenerator [29]. However, no research has been conducted regarding the vector control of grid-connected power electronic converters using DP or ADP-based neural networks.…”

Nested-loop neural network vector control of permanent magnet synchronous motors

“…Adaptive critic designs constitute a class of approximate dynamic programming (ADP) methods that use incremental optimization combined with parametric structures that approximate the optimal cost and the control [22,23]. Both classical DP and ADP methods have been used to train neural networks for a large number of nonlinear control applications, such as steering and controlling the speed of a two-axle vehicle [24], intercepting an agile missile [25], performing auto landing and control of an aircraft [26][27][28], and controlling a turbogenerator [29]. However, no research has been conducted regarding the vector control of PMSMs using DP or ADP-based neural networks.…”