Optimal Control of Conditional Value-at-Risk in Continuous Time

Abstract

We consider continuous-time stochastic optimal control problems featuring Conditional Valueat-Risk (CVaR) in the objective. The major difficulty in these problems arises from timeinconsistency, which prevents us from directly using dynamic programming. To resolve this challenge, we convert to an equivalent bilevel optimization problem in which the inner optimization problem is standard stochastic control. Furthermore, we provide conditions under which the outer objective function is convex and differentiable. We compute the outer objective's value via a Hamilton-Jacobi-Bellman equation and its gradient via the viscosity solution of a linear parabolic equation, which allows us to perform gradient descent. The significance of this result is that we provide an efficient dynamic programming-based algorithm for optimal control of CVaR without lifting the state-space. To broaden the applicability of the proposed algorithm, we propose convergent approximation schemes in cases where our key assumptions do not hold and characterize relevant suboptimality bounds. In addition, we extend our method to a more general class of risk metrics, which includes mean-variance and median-deviation. We also demonstrate a concrete application to portfolio optimization under CVaR constraints. Our results contribute an efficient framework for solving time-inconsistent CVaR-based sequential optimization.

Optimality conditions in variational form for non-linear constrained stochastic control problems

“…In [3,Section 5], a mean-variance portfolio selection problem is considered as well as in [22]. In [19], a gradient-based method is developed for minimization problems of the conditional value at risk, a popular risk-averse cost function. The risk can also be taken into account by considering a constraint of the form G(m 0,Y0,u T ) ≤ 0.…”

Nonlinear PDE Approach to Time-Inconsistent Optimal Stopping

“…We briefly discuss extensions to diffusion processes and more general types of time-inconsistencies in the final section, albeit formally. The ideas are related to recent solutions of mean-variance portfolio optimization, optimal control under certain non-linear risk-measures, and distribution-constrained optimal stopping (see [19,16,2]).…”

General stopping behaviors of naïve and noncommitted sophisticated agents, with application to probability distortion

“…Finally, a sophisticated agent who is able to commit simply solves the problem once at t = 0 and then sticks to the corresponding stopping plan. The problem of the last type, the so-called pre-committed agent, is actually a static (instead of dynamic) problem and has been solved in various contexts, such as optimal stopping under probability distortion in Xu and Zhou (2013), mean-variance portfolio selection in Zhou and Li (2000), optimal stopping with nonlinear constraint on expected time to stop in Miller (2017), and optimal control of conditional Value-at-Risk in Miller and Yang (2017). The goal of this paper is to study the behaviors of the first two types of agents.…”