A class of finite-dimensional numerically solvable McKean-Vlasov control problems

Abstract

We address a class of McKean-Vlasov (MKV) control problems with common noise, called polynomial conditional MKV, and extending the known class of linear quadratic stochastic MKV control problems. We show how this polynomial class can be reduced by suitable Markov embedding to finite-dimensional stochastic control problems, and provide a discussion and comparison of three probabilistic numerical methods for solving the reduced control problem: quantization, regression by control randomization, and regress-later methods. Our numerical results are illustrated on various examples from portfolio selection and liquidation under drift uncertainty, and a model of interbank systemic risk with partial observation.

Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications

“…• Qknn: We used the extension of Algorithm 5 introduced in the paragraph "semi-linear interpolation" of the Section 3.2.2. in [2] and used projection of each state on its k=2nearest neighbors to get an estimate of the value function which is continuous w.r.t. the control variable at each time n = 0, .…”

“…Doing so, one obtains stable estimates of the value function and optimal control, which is desirable. We highlight the fact that this stability procedure cannot be implemented in most of the algorithms proposed in the literature (for example the ones presented in [2] which are based on regress-now, regress-later or quantization).…”

“…Algorithm 5 presents the pseudo-code of an algorithm based on the quantization and k-nearest neighbors methods, called Qknn, which will be the benchmark in all the lowdimensional control problems that will be considered in Section 3 to test NNContPI, Clas-sifPI, Hybrid-Now and Hybrid-Later. Also, comparisons of Algorithm 5 to other well-known algorithms on various control problems in low-dimension are performed in [2], which show in particular that Algorithm 5 works very well to solve low-dimensional control problems. Actually, in our experiments, Algorithm 5 always outperforms the other algorithms based either on regress-now or regress-later methods whenever the dimension of the problem is low enough for Algorithm 5 to be feasible.…”

“…the control variable a, which might cause some stability issues when running Qknn, especially during the optimization procedure (2.9). We refer to Section 3.2.2. in [2] for a detailed presentation of an extension of Algorithm 5 where the estimates of the Q value function Q n is continuous w.r.t. the control variable.…”

A Weak Martingale Approach to Linear-Quadratic McKean–Vlasov Stochastic Control Problems

“…Actually, the candidate w t (x, x ′ ) for the value function should satisfy a Bellman PDE in finite dimension, namely the dimension of (X t , E[X t ]), which is a particular finite dimensional case of the Master equation. This argument of making the McKean-Vlasov control problem finite-dimensional is exploited more generally in [2] where the dependence on the law is through the first p-moments of the state process.…”