A Machine Learning Approach to Adaptive Robust Utility Maximization and Hedging

Abstract

We investigate the adaptive robust control framework for portfolio optimization and loss-based hedging under drift and volatility uncertainty. Adaptive robust problems offer many advantages but require handling a double optimization problem (infimum over market measures, supremum over the control) at each instance. Moreover, the underlying Bellman equations are intrinsically multi-dimensional. We propose a novel machine learning approach that solves for the local saddle-point at a chosen set of inputs and then uses a nonparametric (Gaussian process) regression to obtain a functional representation of the value function. Our algorithm resembles control randomization and regression Monte Carlo techniques but also brings multiple innovations, including adaptive experimental design, separate surrogates for optimal control and the local worst-case measure, and computational speed-ups for the sup-inf optimization. Thanks to the new scheme we are able to consider settings that have been previously computationally intractable and provide several new financial insights about learning and optimal trading under unknown market parameters. In particular, we demonstrate the financial advantages of adaptive robust framework compared to adaptive and static robust alternatives.

Risk-sensitive Markov decision problems under model uncertainty: finite time horizon case

“…In this section, we describe our machine learning based method and present the numerical results for our example. Similarly to [CL19], we discretize the state space the relevant state space in the spirit of the regression Monte Carlo method and adaptive design by creating a random (non-gridded) mesh for the process Y = (X, C). Note that the component X depends on the control process, hence at each time t we randomly select from the set A a value of ϕ t , and we randomly generate a value of Z t+1 , so to simulate the value of X t+1 .…”

“…The main goal of this work is to study finite time horizon risk-sensitive Markovian control problems subject to model uncertainty in a discrete time setup, and to develop a methodology to solve such problems efficiently. The proposed approach hinges on the following main building concepts: incorporating model uncertainty through the adaptive robust paradigm introduced in [BCC + 19] and developing efficient numerical solutions for the obtained Bellman equations by adopting the machine learning techniques proposed in [CL19].…”

Time-inconsistent Markovian control problems under model uncertainty with application to the mean-variance portfolio selection

“…In view of the above mentioned computational challenges, we numerically tackle the adaptive robust stochastic control problem by following the novel method introduced in [CL19]. The key idea of this method is to utilize a non-parametric value function approximation strategy (cf.…”

“…The main goal of this study is to develop a methodology to solve efficiently some time-inconsistent Markovian control problems subject to model uncertainty in a discrete time setup. The proposed approach hinges on the following main building concepts: incorporating model uncertainty through the adaptive robust paradigm introduced in [BCC + 19]; dealing with time-inconsistency of the stochastic control problem at hand by exploding the concept of sub-game perfect strategies as studied in [BM14]; developing efficient numerical solutions for the obtained Bellman equations by adopting the machine learning techniques proposed in [CL19].…”

“…It is welldocumented that solving numerically stochastic control problems subject to model uncertainty is a challenging task, and classical numerical methods can not be successfully applied even to the simplest problems. In [CL19] the authors introduced a method, rooted in the machine learning methodology, to deal with such problems in the context of an adaptive robust, time-consistent, stochastic control problem. In the present work, we apply a similar computational approach for solving the aforementioned mean-variance problem.…”