Cooperating Agents in the Minority Game

Abstract

We consider the Minority Game in which agents compete for resources by striving to be in the minority group. The agents adapt to the environment by reinforcement learning of the preferences of the policies they hold. Diversity of preferences of policies among agents is introduced by adding random biases to the cumulative payoffs of their policies. Agent cooperation becomes increasingly important when diversity increases. §1. Introduction A major interest of statistical physics is to study how the collective behavior of a population results from the interactions of its many components. Besides the natural systems traditionally considered by physicists, artificial systems modeling distributed network control in engineering 1) or financial markets in economics 2) have received much recent attention. While a standard analytical approach is to study their steady state behavior described by the Nash equilibria, 3) it is interesting to consider the dynamics of how the steady state is approached. Of particular interest is the case of a population of heterogeneous agents, which have diversified preferences in decision making. In such cases, the cooperation of agents becomes very important.Specifically, we consider the dynamics of the Minority Game which models the collective behavior of agents simultaneously and adaptively competing for limited resources. 4) Previous work showed that when the complexity of the agents' policies is too low, the system suffers from the maladaptive behavior of the agents, meaning that they prematurely rush to adapt to environment changes in bursts. 5) By introducing diversity to the preference of policies of the agents, we have recently demonstrated that a better system efficiency can be attained. 6) Since agent cooperation is important in this regime, it would be interesting to study the mechanisms by which they adapt mutually, and their effects on the system efficiency.In this paper, we explain the cooperative mechanisms which appear successively when the diversity of the agents' preference of policies increases, as recently proposed in Ref. 7). We will provide experimental evidence of these effects, and sketch their analyses which yield excellent agreement with simulations. §2. The Minority GameThe Minority Game consists of a population of N agents competing in an environment of limited resources, N being odd. 4) Each agent makes a decision + or − at each time step, and the minority group wins. For typical control tasks such as the distribution of shared resources, the decisions + and − may represent two alternative resources, so that less agents utilizing a resource implies more abundance. For ecoat UCSF Library on March 13, 2015 http://ptps.oxfordjournals.org/ Downloaded from