Max-plus summation of Fenchel-transformed semigroups for solution of nonlinear Bellman equations

Abstract

Max-plus methods have been explored for solution of first-order, nonlinear HamiltonJacobi-Bellman partial differential equations (HJB PDEs) and corresponding nonlinear control problems. These methods exploit the max-plus linearity of the associated semigroups. In particular, although the problems are nonlinear, the semigroups are linear in the max-plus sense. These methods have been used successfully to compute solutions. Although they provide certain advantages, they still generally suffer from the curse-of-dimensionality. The natural analog to the Laplace transform in ordinary spaces is the Fenchel transform over max-plus spaces, the range space being referred to as the dual space. One can transform the semigroup operators into operators on the dual space. There are natural operations on the transformed operators which may be used to construct solutions of nonlinear control problems. Natural building blocks correspond to transforms of operators for linear/quadratic problems. In this paper, a method for exploiting operations in the Fenchel transform space is used to develop a method for certain problems such that the computational growth in the most time-consuming portion of the computations can be hugely reduced. Although the curse-of-dimensionality is not entirely avoided, the computational cost reductions are very high for some classes of problems.

Optimal control of unknown nonlinear system under event‐triggered mechanism and identifier‐critic‐actor architecture

“…Definition 1. If u(x) is continuous, u(0) = 0, u(x) can stabilize the system (1), and V is finite, then we define u(x) as admissible in regard to the cost function (2) on Ω, denoted by u ∈ Π(Ω).…”

“…Next, to realize optimal control, we need to design an admissible controller to minimize the cost function (2). For any u ∈ Π(Ω), if the cost function ( 2) is continuously differentiable, then the Lyapunov equation is:…”

“…It obtains the optimal strategy via reverse searching and the optimality principle. 1,2 However, obtaining the optimal controller of nonlinear systems is intractable because the analytical solution of the continuous-time Hamilton-Jacobi-Bellman (HJB) equation is not easy to find. In addition, the "Curse of dimensionality," the reliance on dynamic system models and the contradiction between reverse order search and real-time control also greatly limit the application scope of this method.…”

Convergence and stability results of Zhang neural network solving systems of time-varying nonlinear equations

“…Many numerical algorithms have thus been presented and investigated [1]- [3]. However, for most numerical algorithms, they may not be efficient enough because of their serial-processing nature performed on digital computers [4].…”

Distributed dynamic programming for discrete-time stochastic control, and idempotent algorithms

“…Here, max-plus methods are appropriate for problems with maximizing controllers and vice-versa. These methods include maxplus basis-expansion approaches [1], [2], [6], [7], [11], [15], [18], as well as the more recently developed curse-of-dimensionality-free methods [11], [16], [17]. However, stochastic control problems have eluded idempotent methods.…”