Stochastic Perron's Method for Hamilton--Jacobi--Bellman Equations

Bayraktar, Erhan; Ŝırbu, Mihai

doi:10.1137/12090352x

Cited by 59 publications

(94 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The flexibility on this end allows us to work with a deterministic time grid, resulting in approximation over Markov strategies. Obviously, the analytic part of the proof (the "bump-up/down" argument) is similar to [S14c] or [BS13], but those parts of the proof were already similar to the (purely analytic) arguments of Ishii for viscosity Perron [Ish87].…”

mentioning

confidence: 74%

“…In our particular framework, we use asymptotic stochastic semisolutions (so the Perron method here is asymptotic in the stochastic sense of [S14c] or [BS13]). However, we claim that a similar asymptotic Perron's method can be designed in the analytic framework (see Remark 3.1).…”

Section: Proof: the Asymptotic Perron's Methodmentioning

confidence: 99%

“…One could use Ishii's arguments (see [Ish87]) to construct a viscosity solution and later upgrade its regularity (if possible) for the purpose of verification (as in [JS12]) or employ a modification of Perron's method over stochastic semisolutions if one does not expect smooth solutions (see [BS12], [BS14], and [BS13]). Furthermore, with an appropriate definition of semisolutions in the stochastic sense and a suitable model, one can even treat games and problems with model uncertainty, where dynamic programming is particularly cumbersome (see [S14c], [S14a]).…”

mentioning

confidence: 99%

“…Compared to the so-called stochastic Perron method employed in [S14c] or [BS13], the method we introduce here is quite different. The idea in [S14c] and [BS13] was to use exact semisolutions in the stochastic sense, then do Perron.…”

mentioning

confidence: 99%

“…The idea in [S14c] and [BS13] was to use exact semisolutions in the stochastic sense, then do Perron. In order to do so, the flexibility of stopping rules or stopping times was needed, leading to the possibility to complete the analysis only over general feedback strategies (elementary for the purpose of well posedness of the state equation) or general predictable controls.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Asymptotic Perron's Method and Simple Markov Strategies in Stochastic Games and Control

Ŝırbu¹

2015

SIAM J. Control Optim.

Self Cite

View full text Add to dashboard Cite

We introduce a modification of Perron's method, where semisolutions are considered in a carefully defined asymptotic sense. With this definition, we can show, in a rather elementary way, that in a zero-sum game or a control problem (with or without model uncertainty), the value function over all strategies coincides with the value function over Markov strategies discretized in time. Therefore, there are always discretized Markov ε-optimal strategies (uniform with respect to the bounded initial condition). With a minor modification, the method produces a value and approximate saddle points for an asymmetric game of feedback strategies versus counterstrategies. Introduction.The aim of the paper is to introduce the asymptotic Perron's method, i.e., constructing a solution of the Hamilton-Jacobi-Belman-Isaacs (HJBI) equation as the supremum/infimum of carefully defined asymptotic semisolutions. Using this method we show, in a rather elementary way, that the value functions of zero-sum games/control problems can be (uniformly) approximated by some simple Markov strategies for the weaker player (the player in the exterior of the sup/inf or inf/sup). From this point of view, we can think of the method as an alternative to the shaken coefficients method of Krylov [Kry00] (in the case of only one player, under slightly different technical assumptions) or to the related method of regularization of solutions of HJBIs byŚwiȩch in [Świ96a] and [Świ96b] (for control problems or games in Elliott-Kalton formulation). The method of shaken coefficients has been recently used to study games in Elliott-Kalton formulation in [BN] under a convexity assumption (not needed here).While the result on zero-sum games (under our standing assumptions) is rather new, but certainly expected, the goal of the paper is to present the method. To the best of our knowledge, this modification of Perron's method does not appear in the literature. In addition, we believe that it applies to more general situations than we consider here and using either a stochastic formulation (as in the present work) or an analytic one (see Remark 3.1). Compared to the method of shaken coefficients of Krylov, or to the regularization of solutions byŚwiȩch, the analytic approximation of the value function/solution of HJB by smooth approximate solutions is replaced by the Perron construction. The careful definition of asymptotic semisolutions allows us to prove that such semisolutions work well with Markov strategies. The idea of restricting actions to a deterministic time grid is certainly not new; we just provide a method that works well for such strategies/controls. The arguments display once again the *

show abstract

mentioning

confidence: 74%

Section: Proof: the Asymptotic Perron's Methodmentioning

confidence: 99%