Duality and Approximation of Stochastic Optimal Control Problems under Expectation Constraints

Abstract

We consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is in duality with an optimization problem involving the Lagrange multiplier associated with the constraints. Then by convex analysis techniques, we provide a general existence result and some a priori estimation of the dual optimizers. We further provide a necessary and sufficient optimality condition for the initial constrained control problem. The same results are also obtained for a discrete time constrained control problem. Moreover, under additional regularity conditions, it is proved that the discrete time control problem converges to the continuous time problem, possibly with a convergence rate. This convergence result can be used to obtain numerical algorithms to approximate the continuous time control problem, which we illustrate by two simple numerical examples.

Optimal Control of Diffusion Processes with Terminal Constraint in Law

“…hpxqmpdxq for some function h : R d Ñ R and for all m P PpR d q, we say that the constraint is linear and we recover the problem of stochastic optimal control under expectation constraint (as in [9], [14], [34]). Such problems arise in economy and finance when an agent tries to minimize a cost (maximize a utility function) under constraints on the probability distribution of the final output.…”

“…In Yong and Zhou [44] Chapter 3, necessary optimality conditions are proved in the form of a system of forward/backward stochastic differential equations. More recently the problem with constraints on the law of the process has been studied in Pfeiffer [33] and in Pfeiffer, Tan and Zhou [34]. In these works, the authors prove that the problem can be reduced to a "standard" problem (without terminal constraint) by adding a term involving λ ˚h -in the case where the constraint has the form E rhpX T qs ď 0to the final cost for some optimal Lagrange multiplier λ ˚.…”

“…A dual problem over the Lagrange multipliers associated to the constraints is exhibited using abstract duality results. In Pfeiffer, Tan and Zhou [34], the authors provide necessary and sufficient optimality conditions for problems with multiple equality and inequality expectation constraints with much less restrictions on the data than we do and in a path dependent framework. However [34] needs to assume some controllability condition (Assumption 3.1.ii) and works with a compact control set.…”

“…In Pfeiffer, Tan and Zhou [34], the authors provide necessary and sufficient optimality conditions for problems with multiple equality and inequality expectation constraints with much less restrictions on the data than we do and in a path dependent framework. However [34] needs to assume some controllability condition (Assumption 3.1.ii) and works with a compact control set. In our framework, the corresponding controllability condition would be to assume a priori that there exist some control pα 1 , α 2 q such that EphpX α 1 ,α 2 T qq ă 0.…”

Optimal Stopping with Expectation Constraints

“…As to the stochastic control theory with expectation constraint, Yu et al [23] used the measurable selection argument to obtain a DPP result and applied it to quantitative finance problems with various expectation constraints. Pfeiffer et al [49] took a Lagrange relaxation approach to study a continuous-time stochastic control problem with both inequality-type and equality-type expectation constraints and obtained a duality result by the knowledge of convex analysis. Moreover, for stochastic control problems with state constraints, the stochastic target problems with controlled losses and the related geometric dynamic programming principle, see [16,17,19,56,57,58,20,14,18,15] and etc.…”