Multilevel Simulation Based Policy Iteration for Optimal Stopping--Convergence and Complexity

Abstract

This paper presents a novel approach to reduce the complexity of simulation based policy iteration methods for solving optimal stopping problems. Typically, Monte Carlo construction of an improved policy gives rise to a nested simulation algorithm. In this respect our new approach uses the multilevel idea in the context of the nested simulations, where each level corresponds to a specific number of inner simulations. A thorough analysis of the convergence rates in the multilevel policy improvement algorithm is presented. A detailed complexity analysis shows that a significant reduction in computational effort can be achieved in comparison to the standard Monte Carlo based policy iteration. The performance of the multilevel method is illustrated in the case of pricing a multidimensional American derivative.

Unbiased Multilevel Monte Carlo methods for intractable distributions: MLMC meets MCMC

“…The expected utility under partial information finds widespread applications in computational finance, especially in option pricing Belomestny et al [2015], Zhou et al [2021]. Meanwhile, the difference between full and partial utility…”

“…Estimating the nested expectation is known as a challenging problem in Monte Carlo methods due to its involved structure [Naesseth et al, 2015]. Applications arise in statistics [Giles and Goda, 2019], machine learning [Rainforth et al, 2018], and operation research [Belomestny et al, 2015, Zhou et al, 2021. It is known that the standard plug-in Monte Carlo estimators generally have systematic bias and suboptimal convergence rate.…”

“…The methodology in has been extended to different MCMC algorithms including the Hamiltonian Monte Carlo [Heng and Jacob, 2019] and the pseudo-marginal MCMC [Middleton et al, 2020]. In contrast, the MLMC method (both the non-randomized and randomized version) is shown to be successful in estimating the expectation of SDE solutions [Giles, 2008, Rhee and, option pricing [Belomestny et al, 2015, Zhou et al, 2021, and inverse problems [Beskos et al, 2017, Jasra et al, 2018. When the quantity of interest is E π [f ] for challenging underlying distribution π (in contrast to g(E π [f ]) that we considered here), there already exists similar ideas on combining the unbiased MLMC and MCMC framework on certain problems.…”