Nonlinear Markov Games on a Finite State Space (Mean-field and Binary Interactions)

Abstract

Managing large complex stochastic systems, including competitive interests, when one or several players can control the behavior of a large number of particles (agents, mechanisms, vehicles, subsidiaries, species, police units, etc), say N k for a player k, the complexity of the game-theoretical (or Markov decision) analysis can become immense as N k → ∞. However, under rather general assumptions, the limiting problem as all N k → ∞ can be described by a well manageable deterministic evolution. In this paper we analyze some simple situations of this kind proving the convergence of Nashequilibria for finite games to equilibria of a limiting deterministic differential game.

Evolutionary, mean-field and pressure-resistance game modelling of networks security

“…Remark. For a rigorous explanation (not just the formal description that we provide here) of the Markov chain's convergence to the deterministic process given by (5), see, e.g., Kolokoltsov (2012).…”

Corruption and botnet defense: a mean field game approach

“…It is well known that evolutions of this type can be derived rigorously as the dynamic law of large numbers for the corresponding Markov models of a finite number of players, see detail e.g. in [15] or [16].…”

“…Proposition 3.2. If (21) holds for all j = i with the strict inequality, then for sufficiently large λ and sufficiently small β ij there exists a unique solution to the stationary MFG consistency problem (4) and (6) with the optimal controlû i , the stationary distribution is x I i = x * , x S i = 1 − x * with x * given by (11) and it is stable; the optimal payoffs are given by (15), (16), (18), (19). Conversely, if for all sufficiently large λ there exists a solution to the stationary MFG consistency problem (4) and (6) with the optimal controlû i , then (20) holds.…”

“…

Recently developed toy models for the mean-field games of corruption and botnet defence in cyber-security with three or four states of agents are extended to a more general mean-field-game model with 2d states, d ∈ N. In order to tackle new technical difficulties arising from a larger state-space we introduce new asymptotic regimes, namely small discount and small interaction asymptotics. Moreover, the link between stationary and time-dependent solutions is established rigorously leading to a performance of the turnpike theory in a mean-field-game setting.On the one hand, our model is a performance of the general pressure-and-resistancegame framework of [16] and the nonlinear Markov battles of [15], and on the other hand, it represents a simple example of mean-field-and evolutionary-game modeling of networks. Initiating the development of the latter, we stress already here that two-dimensional arrays of states arise naturally in many situations, one of the dimensions being controlled mostly by the decision of agents (say, the level of tax evasion in the context of inspection games) and the other one by a principal (major player) or evolutionary interactions (say, the level of agents in bureaucratic staircase, the type of a computer virus used by botnet herder, etc).We shall dwell upon two basic interpretations of our model: corrupted bureaucrats playing against the principal (say, governmental representative, also referred in literature as benevolent dictator) or computer owners playing against a botnet herder (which then takes the role of the principal), which tries to infect the computers with viruses.

…”

Linear–Quadratic Mean-Field-Type Games: A Direct Method

“…Most studies illustrated these solution methods in the linear-quadratic game with an infinite number of decision-makers [15][16][17][18][19][20][21]. These works assume indistinguishability within classes, and the cost functions were assumed to be identical or invariant per permutation of decision-makers indexes.…”