Improved Sensitivity Relations in State Constrained Optimal Control

Abstract

Sensitivity relations in optimal control provide an interpretation of the costate trajectory and the Hamiltonian, evaluated along an optimal trajectory, in terms of gradients of the value function. While sensitivity relations are a straightforward consequence of standard transversality conditions for state constraint free optimal control problems formulated in terms of control-dependent differential equations with smooth data, their verification for problems with either pathwise state constraints, nonsmooth data, or for problems where the dynamic constraint takes the form of a differential inclusion, requires careful analysis. In this paper we establish validity of both 'full' and 'partial' sensitivity relations for an adjoint state of the maximum principle, for optimal control problems with pathwise state constraints, where the underlying control system is described by a differential inclusion. The partial sensitivity relation interprets the costate in terms of partial Clarke subgradients of the value function with respect to the state variable, while the full sensitivity relation interprets the couple, comprising the costate and Hamiltonian, as the Clarke subgradient of the value function with respect to both time and state variables. These relations are distinct because, for nonsmooth data, the partial Clarke subdifferential does not coincide with the projection of the (full) Clarke subdifferential on the relevant coordinate space. We show for the first time (even for problems without state constraints) that a costate trajectory can be chosen to satisfy the partial and full sensitivity relations simultaneously. The partial sensitivity relation in this paper is new for state constraint problems, while the full sensitivity relation improves on earlier results in the literature (for optimal control problems formulated in terms of Lipschitz continuous multifunctions), because a less restrictive inward pointing hypothesis is invoked in the proof, and because it is validated for a stronger set of necessary conditions.

A Differential Game with Exit Costs

“…and then, if µ is sufficiently small, we can conclude in the standard way getting the conclusion by contradiction to (18). ⊓ ⊔ Remark 8 As already remarked, the exit cost ψ XY , for simultaneous exit of X and Y , does not play any role in the formulation of the Dirichlet problem (11). Indeed, it can never happen that the simultaneous exit cost ψ XY is a "good" choice for both players (i.e.…”

Viability of an open set for stochastic control systems

“…Let us note that the viability property of the interior of a set has many important applications in optimal control of the deterministic systems under state constraints: regularity of the value function, uniqueness of solutions to Hamilton-Jacobi equations, maximum pri nciple and sensitivity relations were investigated with its help, cf. [9,21]. Having in mind similar applications, it is of a crucial importance to have sufficient conditions for the viability of the interior of K for the stochastic control systems as well.…”

Multifunctions of bounded variation

“…Other examples where there are significant differences in the implications of the two kinds of regularity hypotheses arise in the study of regularity properties of the value function for state constrained optimal control problems [3], validity of necessary conditions of optimality for free-time optimal control problems [6], the interpretation of costate trajectories as gradients of the value function [2] and in more general sensitivity analysis.…”