Zubov’s method for controlled diffusions with state constraints

Abstract

Abstract. We consider a controlled stochastic system in presence of state-constraints. Under the assumption of exponential stabilizability of the system near a target set, we aim to characterize the set of points which can be asymptotically driven by an admissible control to the target with positive probability. We show that this set can be characterized as a level set of the optimal value function of a suitable unconstrained optimal control problem which in turn is the unique viscosity solution of a second order PDE which can thus be interpreted as a generalized Zubov equation.Mathematics Subject Classification (2010). Primary 93B05; Secondary 93E20, 49L25.

A Hamilton-Jacobi-Bellman Approach for the Numerical Computation of Probabilistic State Constrained Reachable Sets

“…Following the approach in [10,19,21], for the regularised problem we have obtained a characterization by a HJB equation with mixed Dirichlet-derivative boundary conditions. We have defined a fully discrete SL approximation scheme and we have proved its convergence to the unique viscosity solution of the equation.…”

“…Aim of this section is to characterize the function ϑ ε as a (viscosity) solution to a suitable HJB equation. For doing this, we closely follow the dynamic programming arguments recently developed in [10,19] for optimal control problems with a cost depending on a running maximum. Therefore, in order to directly use those results in our framework, we will rewrite the optimal control problem (7) by means of the cost functional…”

“…A classical strategy to overcome this difficulty consists in adding an auxiliary variable y that, roughly speaking, gets rid of the non-Markovian component of the cost. This has been originally used in [8] where an approximation technique of the L ∞ -norm is used, whereas in [10,19] the HJB equation is derived without making use of any approximation. Let us introduce the following value function:…”

“…Second, the non commutativity between expectation and maximum operator makes problem (3) not satisfying the natural "Markovian structure" necessary to apply the dynamic programming arguments. We here follow the ideas in [8,10,19,21] and define an auxiliary optimal control problem in an augmented state space and derive the HJB equation for this problem recovering the value function ϑ solution of the original problem at a later stage. The obtained HJB equation is defined in a domain and completed with mixed Dirichlet and oblique derivative boundary conditions.…”

Infinite Horizon Stochastic Optimal Control Problems with Running Maximum Cost

“…The proof can be obtained by a modification of the arguments in [22, Theorem 2.1] which shows how to deal with the derivative conditions; see also [24]. We report here the main steps.…”