Finite Dimensional Approximations of Hamilton--Jacobi--Bellman Equations in Spaces of Probability Measures

Gangbo, Wilfrid; Mayorga, Sergio; Święch, Andrzej

doi:10.1137/20m1331135

Cited by 36 publications

(34 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition convergence with rates can be obtained through FBSDE techniques [7]. However, in this present work we make use of the viscosity solution characterization of the optimal value function which is most related to the recent papers [9] and [15]. In [15], the viscosity solutions of HJB equations in finite-agent deterministic or stochastic optimal control problems are shown to converge to that of a limiting HJB equation in the space of probability measures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mean field control and finite dimensional approximation for regime-switching jump diffusions

Bayraktar¹,

Chakraborty²

2021

Preprint

View full text Add to dashboard Cite

We consider a mean field control problem with regime switching in the state dynamics. The corresponding value function is characterized as the unique viscosity solution of a HJB master equation. We prove that the value function is the limit of a finite agent centralized optimal control problem as the number of agents go to infinity. In the process, we derive the convergence rate and a propagation of chaos result for the optimal trajectory of agent states.

show abstract

Section: Introductionmentioning

confidence: 99%

“…The latter equation is interpreted through Lions' lifting in the L 2 sense. However convergence rates are absent in [15]. But since we rely on our particular viscosity solution structure, we can adopt the ideas in [9] even though the problem addressed there is in the space of probability measures with finite support.…”

Section: Introductionmentioning

confidence: 99%

Mean field control and finite dimensional approximation for regime-switching jump diffusions

Bayraktar¹,

Chakraborty²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Notice however that [17] includes common noise and sets the problem on the torus whereas the present case considers interaction through the controls. The convergence of viscosity solutions was more recently investigated by Gangbo, Mayorga, and Święch [51].…”

Section: Pde Interpretationsmentioning

confidence: 99%

Non-asymptotic convergence rates for mean-field games: weak formulation and McKean--Vlasov BSDEs

Possamaï¹,

Tangpi²

2021

Preprint

View full text Add to dashboard Cite

This work is mainly concerned with the so-called limit theory for mean-field games. Adopting the weak formulation paradigm put forward by Carmona and Lacker [23], we consider a fully non-Markovian setting allowing for drift control, and interactions through the joint distribution of players' states and controls. We provide first a new characterisation of mean-field equilibria as arising from solutions to a novel kind of McKean-Vlasov backward stochastic differential equations, for which we provide a well-posedness theory. We incidentally obtain there unusual existence and uniqueness results for mean-field equilibria, which do not require short-time horizon, separability assumptions on the coefficients, nor Lasry and Lions's monotonicity conditions, but rather smallness conditions on the terminal reward. We then take advantage of this characterisation to provide non-asymptotic rates of convergence for the value functions and the Nash-equilibria of the N -player version to their mean-field counterparts, for general open-loop equilibria. This relies on new backward propagation of chaos results, which are of independent interest. An appropriate reformulation of our approach also allows us to treat closed-loop equilibria, and to obtain convergence results for the master equation associated to the problem.

show abstract

“…In the first-order case a kind of equivalence result between two related definitions is provided in [27,Theorem 4.4]. In a second-order semi-linear case some results in this direction are provided in [23,Section 5]. We are not aware of any results on the fully non-linear second-order case.…”

Section: Master Bellman Equationmentioning

confidence: 99%

“…In fact, it gave rise to a stochastic calculus on the space of probability measures, and in particular to an Itô formula (chain rule) for maps defined on the Wasserstein space (we recall it in our Theorem 3.3), which allows to relate the value function of the control problem to the Bellman equation (we recall it in our Theorem 3.8). Regarding the relation between partial differential equations adopting the derivatives introduced by Lions (as in the present paper) and equations using notions of differentiability as those adopted in optimal transport theory, we mention results in this direction in the first-order case in [27] and in a second-order semi-linear case in [23] (see also Remark 3.6).…”

Section: Introductionmentioning

confidence: 99%

Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions

Cosso¹,

Gozzi²,

Kharroubi³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study the Bellman equation in the Wasserstein space arising in the study of mean field control problems, namely stochastic optimal control problems for McKean-Vlasov diffusion processes. Using the standard notion of viscosity solution à la Crandall-Lions extended to our Wasserstein setting, we prove a comparison result under general conditions, which coupled with the dynamic programming principle, implies that the value function is the unique viscosity solution of the Master Bellman equation. This is the first uniqueness result in such a second-order context. The classical arguments used in the standard cases of equations in finite-dimensional spaces or in infinite-dimensional separable Hilbert spaces do not extend to the present framework, due to the awkward nature of the underlying Wasserstein space. The adopted strategy is based on finite-dimensional approximations of the value function obtained in terms of the related cooperative n-player game, and on the construction of a smooth gauge-type function, built starting from a regularization of a sharpe estimate of the Wasserstein metric; such a gauge-type function is used to generate maxima/minima through a suitable extension of the Borwein-Preiss generalization of Ekeland's variational principle on the Wasserstein space.

show abstract

Finite Dimensional Approximations of Hamilton--Jacobi--Bellman Equations in Spaces of Probability Measures

Cited by 36 publications

References 35 publications

Mean field control and finite dimensional approximation for regime-switching jump diffusions

Mean field control and finite dimensional approximation for regime-switching jump diffusions

Non-asymptotic convergence rates for mean-field games: weak formulation and McKean--Vlasov BSDEs

Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions

Contact Info

Product

Resources

About