Approximations and computational methods for Optimal Stopping and Stochastic Impulsive Control problems

Abstract

The paper treats a computational method for the Optimal Stopping and Stochastic Impulsive Control problem for a diffusion. In the latter problem control acts only intermittently since there is a basic positive "transaction" cost to be paid at each instant that the control acts. For each h > 0, a controlled Markov chain is constructed, whose continuous time interpolations are a natural approximation to the diffusion, for both the optimal stopping and impulsive control situations. The solutions to the optimal stopping and impulsive control problems for the chains are relatively easy to obtain by using standard procedures, and they converge to the solutions of the corresponding problems for the diffusion models as h-,0.

Impulse Control of Interest Rates

“…The algorithm converges to the optimal value function at the bottom of Figure 4 and the optimal policy is described by d D U u = 0 0334 0.0614, 0.0853, 0.1343 . Although one can discretize the problem using the finite difference schemes proposed in Kushner (1976) and then brute force the solution using value or policy iteration, this methodology is not expected to perform well on impulse control problems because of the discontinuities in the state evolution that translate to insufficient smoothness in the value function for accelerated convergence. However, for illustration, comparing our method with that of using policy iteration on the discretized Markov chain, our method converges after 22 iterations in 0.176 seconds on a 10,000 point grid, whereas policy iteration in the controlled Markov chain case converges after 703 iterations in 1,328 seconds on the same grid.…”