Improved order 1/4 convergence for piecewise constant policy approximation of stochastic control problems

Abstract

In N. V. Krylov, Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies, Electron. J. Probab., 4(2), 1999, it is proved under standard assumptions that the value functions of controlled diffusion processes can be approximated with order 1/6 error by those with controls which are constant on uniform time intervals. In this note we refine the proof and show that the provable rate can be improved to 1/4, which is optimal in our setting. Moreover, we demonstrate the improvements this implies for error estimates derived by similar techniques for approximation schemes, bringing these in line with the best available results from the PDE literature.Here we use the notation ϕ a (·, ·) = ϕ(·, ·, a) for any a ∈ A and function ϕ. For a given terminal cost function g and running cost f , the optimal control problem consists of maximizing over α ∈ A the expected total cost( 1.2)The indices on the expectation E indicate that the law of the process depends on the starting point and control. Finally, the value function of the optimal control problem is defined byWe consider the following set of assumptions:(H1) A is a compact set;d×p are continuous functions. For ϕ ∈ {b, σ}, there exists C 0 ≥ 0 such that for every t, s ∈ [0, T ], x, y ∈ R d , a ∈ A: |ϕ(t, x, a) − ϕ(s, y, a)| ≤ C 0 |x − y| + |t − s| 1/2 and |ϕ(t, x, a)| ≤ C 0 ; (H3) g : R d → R and f : [0, T ] × R d × A → R are continuous functions. There exists C 1 ≥ 0 such that for every t, s ∈ [0, T ], x, y ∈ R d , a ∈ A:|g(x) − g(y)| ≤ C 1 |x − y|, |f (t, x, a) − f (s, y, a)| ≤ C 1 |x − y| + |t − s| 1/2 and |f (t, x, a)| ≤ C 1 .

Duality-based a posteriori error estimates for some approximation schemes for optimal investment problems

“…An upper bound of order 1/6 for the error related to this approximation was first obtained by Krylov in [20]. Recently, this estimate has been improved to the order 1/4 in [16], so that one has…”

“…for some constant C ≥ 0. We point out that the results in [20] and [16] require some additional assumptions on the coefficients and do not directly apply to problem (2.1) to (2.2). It is possible that analogous estimates hold also in the setting of the present paper, but since we do not make use of (3.3) here, we did not check this point in detail.…”

“…Indeed, a main objective of the present paper is to by-pass the estimate (3.3), which turns out to be a bottleneck in the provable approximation order, while still using the piecewise constant policy approximation itself by building an approximation to v h . The more important observation from [16] is therefore that a better order than 1/4 is not provable in the general case of Lipschitz viscosity solutions. Then no matter how precise the estimates obtained for the error of the final approximation to v h are, without any further information the upper error bounds to v cannot be more accurate than O(h 1/4 ).…”

Probabilistic error analysis for some approximation schemes to optimal control problems

“…Under assumptions (H1)-(H3), an upper bound of order 1/6 for the error related to this approximation was first obtained by Krylov in [17]. Recently, this estimate has been improved to the order 1/4 in [14], so that one has…”

“…This, together with (5.1), gives estimates for the lower bound of order O(h + |∆x| 2 h ). It is also shown in [14] that 0 ≤ v − v h ≤ Ch holds if v h is sufficiently smooth. For |∆x| ∼ h this leads to error estimates of order 1.…”