Competition, efficiency and collective behavior in the “El Farol” bar model

Cara, M. A. R. de; Plá, O.; Guinea, F.

doi:10.1007/s100510050843

Cited by 45 publications

(46 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was found, that the properties of the game depend on the memory length M and number of players N through the scaling variable α = 2 M /N [4,6,7]. The Minoriry game was thoroughly studied both numerically and analytically [8][9][10][11][12][13][14][15][16][17][18][19] along with the study of the original bar attendance problem [20,21] and various modifications of the Minority game [22][23][24][25][26][27][28][29] An important role is attributed to the observation that the dynamics of the memorized pattern and the strategies' scores are in certain regimes decoupled [30][31][32] and that the thermal noise can ge introduced in the players' decisions [33][34][35].…”

Section: Introductionmentioning

confidence: 99%

Dynamical spin-glass-like behavior in an evolutionary game

Slanina

Zhang

2001

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Dynamical spin-glass-like behavior in an evolutionary game

Slanina

Zhang

2001

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

show abstract

“…The multi-night bar problem is an abstraction of congestion games (and a variant of the El Farol bar problem [5]) which have been extensively studied [1,5,9,7,14]. In this version of the congestion problem, each agent has to determine which day in the week to attend a bar.…”

Section: Congestion Problemsmentioning

confidence: 99%

“…This leads to modifying Eq. (9), that for agent i and action b leads to a weighted team reward D WT :…”

Section: Ant Reward With Weighted Teamsmentioning

confidence: 99%

Learning From Actions Not Taken in Multiagent Systems

Tumer

Khani

2009

Advs. Complex Syst.

View full text Add to dashboard Cite

In large cooperative multiagent systems, coordinating the actions of the agents is critical to the overall system achieving its intended goal. Even when the agents aim to cooperate, ensuring that the agent actions lead to good system level behavior becomes increasingly difficult as systems become larger. One of the fundamental difficulties in such multiagent systems is the slow learning process where an agent not only needs to learn how to behave in a complex environment, but also needs to account for the actions of other learning agents. In this paper, we present a multiagent learning approach that significantly improves the learning speed in multiagent systems by allowing an agent to update its estimate of the rewards (e.g. value function in reinforcement learning) for all its available actions, not just the action that was taken. This approach is based on an agent estimating the counterfactual reward it would have received had it taken a particular action. Our results show that the rewards on such "actions not taken" are beneficial early in training, particularly when only particular "key" actions are used. We then present results where agent teams are leveraged to estimate those rewards. Finally, we show that the improved learning speed is critical in dynamic environments where fast learning is critical to tracking the underlying processes.

show abstract

“…Researchers have also studied the dynamics of the bar problem to investigate economic properties like competition, cooperation and collective behavior and especially their relationship to market e ciency 64,141,232]. 20 …”

Section: El Farol Bar Problemmentioning

confidence: 99%

“…To concentrate more precisely on the issue of COIN initialization, we ran subsequent experiments on variants of Arthur's famous \El Farol bar problem" (see Section 3). To facilitate the analysis we modi ed Arthur's original problem to be more general, and since we were not interested in directly comparing our results to those in the literature, we used a more conventional (and arguably \dumber")machine learning algorithm than the ones investigated in 4,47,50,64].…”

Section: Coin Initializationmentioning

confidence: 99%

Theory of Collective Intelligence

Wolpert

2004

Collectives and the Design of Complex Systems

View full text Add to dashboard Cite

This paper surveys the emerging science of how to design a \COllective INtelligence" (COIN). A COIN is a large multi-agent system where: i) There is little to no centralized, personalized communication and/or control. ii) There is a provided world utility function that rates the possible histories of the full system.In particular, we a r e i n terested in COINs in which e a c h a g e n t runs a reinforcement learning (RL) algorithm. The conventional approach to designing large distributed systems to optimize a world utility does not use agents running RL algorithms. Rather, that approach begins with explicit modeling of the dynamics of the overall system, followed by detailed hand-tuning of the interactions between the components to ensure that they \cooperate" as far as the world utility i s concerned. This approach is labor-intensive, often results in highly nonrobust systems, and usually results in design techniques that have limited applicability. In contrast, we w i s h t o s o l v e the COIN design problem implicitly, via the \adaptive" character of the RL algorithms of each of the agents. This approach i n troduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we d o n o t have phenomena like t h e tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research speci cally concentrating on the COIN design problem has already resulted in successes in arti cial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed science of COIN design may provide much insight into other already established scienti c elds, such as economics, game theory, and population biology.

show abstract

Competition, efficiency and collective behavior in the “El Farol” bar model

Cited by 45 publications

References 7 publications

Dynamical spin-glass-like behavior in an evolutionary game

Dynamical spin-glass-like behavior in an evolutionary game

Learning From Actions Not Taken in Multiagent Systems

Theory of Collective Intelligence

Contact Info

Product

Resources

About