A Study on Numerical Solutions of Hamilton-Jacobi-Bellman Equations Based on Successive Approximation Approach

Abstract

This paper presents a numerical approach to solve the Hamilton-Jacobi-Bellman (HJB) equation, which arises in nonlinear optimal control. In this approach, we first use the successive approximation to reduce the HJB equation, a nonlinear partial differential equation (PDE), to a sequence of linear PDEs called a generalized-Hamilton-Jacobi-Bellman (GHJB) equation. Secondly, the solution of the GHJB equation is decomposed by basis functions whose coefficients are obtained by the collocation method. This step is conducted by solving quadratic programming under the constraints which reflect the conditions that the value function must satisfy. This approach enables us to obtain a stabilizing solution of problems with strong nonlinearity. The application to swing up and stabilization control of an inverted pendulum illustrates the effectiveness of the proposed approach.