Flexible forward improvement iteration for infinite time horizon Markovian optimal stopping problems

Abstract

In this paper, we propose an extension of the forward improvement iteration algorithm, originally introduced in Irle [2006] and recently reconsidered in Miclo and Villeneuve [2021]. The main new ingredient is a flexible window parameter describing the look-ahead distance in the improvement step. We consider the framework of a Markovian optimal stopping problem in discrete time with random discounting and infinite time horizon. We prove convergence and show that the additional flexibility may significantly reduce the runtime.