Submodular Mean Field Games: Existence and Approximation of Solutions

Dianetti, Jodi; Ferrari, Giorgio; Fischer, Markus; Nendel, Max

doi:10.48550/arxiv.1907.10968

Cited by 4 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The behavior of learning dynamics in submodular games is more involved [20]. However, following [38] and [9], we obtain guarantees on the stability of certain learning processes.…”

Section: Long-term Property Of the Gamementioning

confidence: 90%

See 1 more Smart Citation

Herd Behaviors in Epidemics: A Dynamics-Coupled Evolutionary Games Approach

Liu¹,

Zhao²,

Zhu³

2021

Preprint

View full text Add to dashboard Cite

The recent COVID-19 pandemic has led to an increasing interest in the modeling and analysis of infectious diseases. The pandemic has made a significant impact on the way we behave and interact in our daily life. The past year has witnessed a strong interplay between human behaviors and epidemic spreading. In this paper, we propose an evolutionary game-theoretic framework to study the coupled evolutions of herd behaviors and epidemics. Our framework extends the classical degree-based mean-field epidemic model over complex networks by coupling it with the evolutionary game dynamics. The statistically equivalent individuals in a population choose their social activity intensities based on the fitness or the payoffs that depend on the state of the epidemics. Meanwhile, the spreading of the infectious disease over the complex network is reciprocally influenced by the players' social activities. We analyze the coupled dynamics by studying the stationary properties of the epidemic for a given herd behavior and the structural properties of the game for a given epidemic process. The decisions of the herd turn out to be strategic substitutes. We formulate an equivalent finite-player game and an equivalent network to represent the interactions among the finite populations. We develop structurepreserving approximation techniques to study time-dependent properties of the joint evolution of the behavioral and epidemic dynamics. The resemblance between the simulated coupled dynamics and the real COVID-19 statistics in the numerical experiments indicates the predictive power of our framework.

show abstract

“…The behavior of learning dynamics in submodular games is more involved [20]. However, following [38] and [9], we obtain guarantees on the stability of certain learning processes.…”

Section: Long-term Property Of the Gamementioning

confidence: 90%

“…Note that the maximal and the minimal points x * max and x * min do not, in general, correspond to the equilibrium points where the payoffs of players reach the maximum and the minimum, respectively [9]. Corollary 1 enables the monotone convergence to NE of the evolutionary dynamics of the form:…”

Section: Long-term Property Of the Gamementioning

confidence: 99%

Herd Behaviors in Epidemics: A Dynamics-Coupled Evolutionary Games Approach

Liu¹,

Zhao²,

Zhu³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, IRL on this MDP rationalises expert behaviours by finding reward function under which the expert policy maximises the population's societal reward. However, a ERMFNE (also MFNE) is not unique in general, and does not necessarily always maximises the population's societal reward [3,15,12,11]. In particular, Subramanian and Mahajan [39] analyses the MFNE that maximises societal reward, where authors call such MFNE the mean-field social-welfare optimal (MF-SO).…”

Section: Reducing Mfg To Mdpmentioning

confidence: 99%

“…As MaxEnt IRL operates at the population level, it presupposes that agents in the MFG are cooperative, i.e., they aim to optimise a common "societal reward". This does not necessarily align with the interest of each individual agent, since a MFNE (or ERMFNE) does not always maximise the population's societal reward if multiple equilibria exist [39,3,15,12,11]. In fact, the MFNE that maximises societal reward is defined as the so-called mean-field social-welfare optimal (MF-SO) in [39, Definition 2.2], where authors gives a detailed explanation about the difference between MFNE and MF-SO.…”

mentioning

confidence: 99%

Agent-Level Maximum Entropy Inverse Reinforcement Learning for Mean Field Games

Chen¹,

Zhang²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Mean field games (MFG) facilitate the otherwise intractable reinforcement learning (RL) in large-scale multi-agent systems (MAS), through reducing interplays among agents to those between a representative individual agent and the mass of the population. While, RL agents are notoriously prone to unexpected behaviours due to reward mis-specification. This problem is exacerbated by an expanding scale of MAS. Inverse reinforcement learning (IRL) provides a framework to automatically acquire proper reward functions from expert demonstrations. Extending IRL to MFG, however, is challenging due to the complex notion of mean-field-type equilibria and the coupling between agent-level and population-level dynamics. To this end, we propose mean field inverse reinforcement learning (MFIRL), a novel model-free IRL framework for MFG. We derive the algorithm based on a new equilibrium concept that incorporates entropy regularization, and the maximum entropy IRL framework. Experimental results on simulated environments demonstrate that MFIRL is sample efficient and can accurately recover the ground-truth reward functions, compared to the state-of-the-art method.

show abstract

“…Some other techniques exist: Common noise models with a special structure of translation-invariance permit a reduction to the case without common noise [53]. See also [30] for a construction of strong MFE for so-called submodular models, via order-theoretic rather than topological methods.…”

Section: Introductionmentioning

confidence: 99%

Closed-loop convergence for mean field games with common noise

Lacker¹,

Flem²

2021

Preprint

View full text Add to dashboard Cite

This paper studies the convergence problem for mean field games with common noise. We define a suitable notion of weak mean field equilibria, which we prove captures all subsequential limit points, as n → ∞, of closed-loop approximate equilibria from the corresponding n-player games. This extends to the common noise setting a recent result of the first author, while also simplifying a key step in the proof and allowing unbounded coefficients and non-i.i.d. initial conditions. Conversely, we show that every weak mean field equilibrium arises as the limit of some sequence of approximate equilibria for the n-player games, as long as the latter are formulated over a broader class of closed-loop strategies which may depend on an additional common signal.

show abstract

Submodular Mean Field Games: Existence and Approximation of Solutions

Cited by 4 publications

References 21 publications

Herd Behaviors in Epidemics: A Dynamics-Coupled Evolutionary Games Approach

Herd Behaviors in Epidemics: A Dynamics-Coupled Evolutionary Games Approach

Agent-Level Maximum Entropy Inverse Reinforcement Learning for Mean Field Games

Closed-loop convergence for mean field games with common noise

Contact Info

Product

Resources

About