A power penalty method for discrete HJB equations

Abstract

We develop a power penalty approach to the discrete Hamilton-Jacobi-Bellman (HJB) equation in R N in which the HJB equation is approximated by a nonlinear equation containing a power penalty term. We prove that the solution to this penalized equation converges to that of the HJB equation at an exponential rate with respect to the penalty parameter when the control set is finite and the coefficient matrices are M -matrices. Examples are presented to confirm the theoretical findings and to show the efficiency of the new method.

Solution method for discrete double obstacle problems based on a power penalty approach

“…First, it is easy to implement and robust to the scale of the problem. Second, it can be applied to more complicated problems, such as nonlinear complementarity problem [13], Hamilton-Jacobi-Bellman problem [16], etc. Third, the accuracy of the penalty method can be simply controlled by the penalty parameters with an exponential convergence rate.…”

“…We comment that in [16] we propose a power penalty approach to the discrete Hamilton-Jacobi-Bellman equation. However, the discrete double obstacle problem (1) is an Hamilton-Jacobi-Bellman-Issac equation, or an Hamilton-Jacobi-Bellman complementarity problem, rather than an Hamilton-Jacobi-Bellman equation.…”

“…However, the discrete double obstacle problem (1) is an Hamilton-Jacobi-Bellman-Issac equation, or an Hamilton-Jacobi-Bellman complementarity problem, rather than an Hamilton-Jacobi-Bellman equation. Thus, the results developed in [16] do not apply to Problem 1 any longer. As can be seen below, both the formulation and analysis for the discrete double obstacle problem (1) are different from that in [16].…”

“…Thus, the results developed in [16] do not apply to Problem 1 any longer. As can be seen below, both the formulation and analysis for the discrete double obstacle problem (1) are different from that in [16]. We also comment that the interior penalty approach to the discrete mixed complementarity problem, developed in [18], is a promising method to solve (1).…”

“…We then apply the classic Newton method to solve (16), which yields the following algorithm. For clarity, we omit the subscript λ of x λ in the algorithm.…”