Local Minimization Algorithms for Dynamic Programming Equations

Abstract

The numerical realization of the dynamic programming principle for continuous-time optimal control leads to nonlinear Hamilton-Jacobi-Bellman equations which require the minimization of a nonlinear mapping over the set of admissible controls. This minimization is often performed by comparison over a finite number of elements of the control set. In this paper we demonstrate the importance of an accurate realization of these minimization problems and propose algorithms by which this can be achieved effectively. The considered class of equations includes nonsmooth control problems with 1 -penalization which lead to sparse controls.

Sparse and switching infinite horizon optimal controls with mixed-norm penalizations

“…This scheme has also been applied to the solution of sparse optimal feedback control problems in [1,13]. In the case p = q = 1 the minimization operation in (6.27) can be realized by means of semismooth Newton methods as [20].…”

“…The Hamilton-Jacobi-Bellman equation for impulse and switching controls was discussed in [5,28]. The synthesis of sparse feedback laws via dynamic programming has been studied in [13,20,1]. In the context of partial differential equations optimal control of systems switching among different modes were analysed in [16,17], problems with convex switching enhancing functionals were investigated in [11], and problems with nonconvex switching penalization in [12].…”

(Sub)Optimal feedback control of mean field multi-population dynamics

“…We propose a numerical scheme for system (3.26), which extend the approach proposed in Albi et al (2017a), consisting of two main parts. First, an off-line procedure for the synthesis of the feedback map (2.15) via the solution of (2.19), approximated with a semi-Lagrangian, policy iteration scheme Alla et al (2015); Kalise et al (2016). Second, the direct solution of the Boltzmann system (3.23) via a modified Direct Simulation Monte-Carlo method (DSMC), Bobylev and Nanbu (2000); Pareschi and Toscani (2013).…”

A HJB-POD feedback synthesis approach for the wave equation

“…, u m }. This, of course, introduces an additional discretization error which could be avoided working in a continuous space as in [12] or by a descent method without derivatives (like Brent's algorithm). The discretization error can be reduced by an iterative bisection method described in [7].…”