Error Estimates of Penalty Schemes for Quasi-Variational Inequalities Arising from Impulse Control Problems

Abstract

This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the schemes are half-order accurate for HJBQVIs with Lipschitz coefficients, and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse controls based on the error estimates and the penalized solutions. The penalty schemes and convergence results are then extended to HJBQVIs with possibly negative impulse costs. We also demonstrate the convergence of monotone discretizations of the penalized equations, and establish that policy iteration applied to the discrete equation is monotonically convergent with an arbitrary initial guess in an infinite dimensional setting. Numerical examples for infinite-horizon optimal switching problems are presented to illustrate the effectiveness of the penalty schemes over the conventional direct control scheme.

A Fixed-Point Policy-Iteration-Type Algorithm for Symmetric Nonzero-Sum Stochastic Impulse Control Games

“…However, it is trivial to see that: WCDD L 0 -matrix ⇐⇒ WCDD L-matrix. 35 The equivalence is due to (A k ) being substochastic matrices, which implies that B k ∞ ≤ 1 for all k.…”

“…This is not so for policy iteration, which converges superlinearly (see[11, Thm.3.4] and[34, Sect.5]), making it a better suited choice in this case (in terms of runtime as well). Furthermore, it has been demonstrated in[35][Sect.7] that the number of steps required for penalized…”

“…(H2) and Thm.4.3], and a counterexample of convergence in its absence in [4, Ex.4.9] 15. See[34,35] for a novel, globally convergent, combined policy iteration and penalization scheme for general impulse control HJBQVIs 16. In [17, Rmk.6.5], assumptions are given to guarantee the zero-sum analogous toM i V i ≤ H i V i ,with a strict inequality if such assumptions are slightly strengthened.…”

Uncertain Execution in Order-Driven Markets

“…The convergence of these algorithms in our current framework remains to be studied. For penalty schemes of viscosity solutions to HJBQVIs (with enough regularity, e.g., Lipschitz continuity or semiconvexity) see Reisinger and Zhang (2020).…”

A Neural Network-Based Policy Iteration Algorithm with Global $$H^2$$-Superlinear Convergence for Stochastic Games on Domains

“…A nonconvex HJBI equation as above also arises from a penalty approximation of hybrid control problems involving continuous controls, optimal stopping and impulse controls, where the HJB (quasi-)variational inequality can be reduced to an HJBI equation by penalizing the difference between the value function and the obstacles (see e.g. [27,39,40,47]). As (1.1) in general cannot be solved analytically, it is important to construct effective numerical schemes to find the solution of (1.1) and its derivatives.…”