Optimal control policies for evolutionary dynamics with environmental feedback

Abstract

We study a dynamical model of a population of cooperators and defectors whose actions have long-term consequences on environmental "commons" -what we term the "resource". Cooperators contribute to restoring the resource whereas defectors degrade it. The population dynamics evolve according to a replicator equation coupled with an environmental state. Our goal is to identify methods of influencing the population with the objective to maximize accumulation of the resource. In particular, we consider strategies that modify individual-level incentives. We then extend the model to incorporate a public opinion state that imperfectly tracks the true environmental state, and study strategies that influence opinion. We formulate optimal control problems and solve them using numerical techniques to characterize locally optimal control policies for three problem formulations: 1) control of incentives, and control of opinions through 2) propaganda-like strategies and 3) awareness campaigns. We show numerically that the resulting controllers in all formulations achieve the objective, albeit with an unintended consequence. The resulting dynamics include cycles between low and high resource statesa dynamical regime termed an "oscillating tragedy of the commons". This outcome may have desirable average properties, but includes risks to resource depletion. Our findings suggest the need for new approaches to controlling coupled populationenvironment dynamics.

Social Welfare Maximization and Conformism via Information Design in Linear-Quadratic-Gaussian Games

“…Information design is one form of intervention mechanism to influence behavior of strategic agents [6,28]. Other intervention mechanisms include providing financial incentives [6,29,30], system utility design [31,32,33,34,35], and nudging or player control during learning dynamics [36,37,38,39,40]. Financial incentives include taxations [6] or rewards [29] that directly modify the utility of the agents.…”

“…Other intervention mechanisms include providing financial incentives [6,29,30], system utility design [31,32,33,34,35], and nudging or player control during learning dynamics [36,37,38,39,40]. Financial incentives include taxations [6] or rewards [29] that directly modify the utility of the agents. System utility design is motivated by the use of game theoretic learning dynamics in technological multi-agent systems in which individual payoffs are assigned/coded by a system designer prior to the dynamics so that the learning dynamics converge to a desired operating conditions.…”

Limit Cycles Analysis and Control of Evolutionary Game Dynamics with Environmental Feedback

“…We also emphasize that research on how to design reasonable and effective control policies to achieve expected system behaviors in evolutionary games has attracted increasing attention [17], [37], [25], [27]. Specifically, using game-environment feedback, [21] considers optimal control problems where control takes the form of the incentives and opinions respectively. The adopted control policies can maximize accumulated resource over time.…”

The Impact of Irrational Behaviours in the Optional Prisoner's Dilemma with Game-Environment Feedback

“…Game-environment feedback, or in general coevolutionary game theory, has generated increasing interest in the control community, where the design of adequate control policies play a crucial role in achieving a specific system behaviour 20,21 . An example of the use of game-environment feedback in control is the work by Paarporn et al where the controlled system exhibits oscillatory behaviours due to deprecation of the environmental resource 22 . Examples from such a wide range of research areas show that an understanding of the dynamics in the presence of game-environment feedback is a crucial aspect for a deeper awareness of complex real-life systems.…”