Nonzero-Sum Stochastic Games and Mean-Field Games with Impulse Controls

Abstract

We consider a general class of nonzero-sum N-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limiting situation when N goes to infinity, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, there exists a unique NE solution to the MFG, which is an ϵ-NE approximation to the N-player game, with [Formula: see text]. As an example, we analyze in detail a class of two-player stochastic games which extends the classical cash management problem to the game setting. In particular, we present numerical analysis for the cases of the single player, the two-player game, and the MFG, showing the impact of competition on the player’s optimal strategy, with sensitivity analysis of the model parameters.

Nash equilibria in nonzero-sum differential games with impulse control

“…Our work is closely related to the impulse games studied in [28,12,13], and [29] with a feedback information structure where, however, all players are assumed to make discrete-time interventions in the continuous-time stochastic processes. To illustrate, [28] studied a specialized pollution control game between a government that determines the regulatory constraints on emissions and a (representative) firm that takes discrete-time actions to expand its capacity.…”

“…Reference [13] extended their two-player model to an -player setting and analyzed the corresponding mean-field game. In [30], a game problem between an impulse player and a stopper is solved using the QVIs.…”

“…Remark 8. In impulse games studied in [12] and [13], the system of QVIs for any player has an additional intervention operator to account for impulses by the other player(s), while the Hamiltonian operator is not an explicit function of the strategies of other player(s). In our game problem, the Hamiltonian operator of Player 2 depends on the strategies of Player 1, which in turn continuously affects the continuation and intervention sets of Player 2.…”

“…The novel feature of our model is that Player 1 can continuously change both the state trajectory and Player 2's continuation set, which is a collection of all time and state vectors for which it is optimal for Player 2 not to intervene in the system. This feature differentiates our work from the existing literature on differential games with impulse control (see [12] and [13]), where the continuous evolution of the state is exogenously given and all players shift the state from one level to another at discrete time instants. Since the FNE strategies obtained by using the verification theorem are a function of the current time and state pairs, they are strongly time consistent.…”

Stationary Discounted and Ergodic Mean Field Games of Singular Control

“…It thus follows that one can hope to determine the explicit structure of mean field equilibria only in specific settings, as when the game has a linear-quadratic structure (see, e.g, Bensoussan et al (2016) and references therein), or when the problem is stationary and the interaction is through moments of the population's state with respect to the stationary distribution (cf. Basei et al (2020) and Cao and Guo (2020), among others).…”