Adaptive Load Balancing: A Study in Multi-Agent Learning

Schaerf, Andrea; Shoham, Yoav; Tennenholtz, Moshe

doi:10.1613/jair.121

Cited by 133 publications

(97 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Solipsistic Agents: MAS's with solipsistic agents have been successfully applied to 9 a multitude of problems 60,112,122,227,233]. Generally, these schemes use RL algorithms similar to those discussed in Section 3.1.1.…”

Section: Reinforcement Learning-based Multi-agent Systemsmentioning

confidence: 99%

Theory of Collective Intelligence

Wolpert

2004

Collectives and the Design of Complex Systems

View full text Add to dashboard Cite

This paper surveys the emerging science of how to design a \COllective INtelligence" (COIN). A COIN is a large multi-agent system where: i) There is little to no centralized, personalized communication and/or control. ii) There is a provided world utility function that rates the possible histories of the full system.In particular, we a r e i n terested in COINs in which e a c h a g e n t runs a reinforcement learning (RL) algorithm. The conventional approach to designing large distributed systems to optimize a world utility does not use agents running RL algorithms. Rather, that approach begins with explicit modeling of the dynamics of the overall system, followed by detailed hand-tuning of the interactions between the components to ensure that they \cooperate" as far as the world utility i s concerned. This approach is labor-intensive, often results in highly nonrobust systems, and usually results in design techniques that have limited applicability. In contrast, we w i s h t o s o l v e the COIN design problem implicitly, via the \adaptive" character of the RL algorithms of each of the agents. This approach i n troduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we d o n o t have phenomena like t h e tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research speci cally concentrating on the COIN design problem has already resulted in successes in arti cial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed science of COIN design may provide much insight into other already established scienti c elds, such as economics, game theory, and population biology.

show abstract

Section: Reinforcement Learning-based Multi-agent Systemsmentioning

confidence: 99%

Theory of Collective Intelligence

Wolpert

2004

Collectives and the Design of Complex Systems

View full text Add to dashboard Cite

show abstract

“…(3) The exploration rate, erate, which determines the probability with which the Agent will ignore its trust ratings and send a bid to a random SCP Agent, where 0 represents no exploration of the market and 1.0 causes the SSP Agent to always select suppliers at random. The exploration rate parameter addresses a practical problem familiar in the reinforcement learning paradigm, that of balancing the advantages to be gained from trading with known and already trusted partners with the opportunity to discover better partners from the larger pool ( [21]). …”

Section: Ssp Allocator Functionmentioning

confidence: 99%

Experiments in Building Experiential Trust in a Society of Objective-Trust Based Agents

Witkowski

Artikis

Pitt

2001

Trust in Cyber-Societies

View full text Add to dashboard Cite

Abstract. In this paper we develop a notion of "objective trust" for Software Agents, that is trust of, or between, Agents based on actual experiences between those Agents. Experiential objective trust allows Agents to make decisions about how to select other Agents when a choice has to be made. We define a mechanism for such an "objective Trust-Based Agent" (oTB-Agent), and present experimental results in a simulated trading environment based on an Intelligent Networks (IN) scenario. The trust one Agent places in another is dynamic, updated on the basis of each experience. We use this to investigate three questions related to trust in Multi-Agent Systems (MAS), first how trust affects the formation of trading partnerships, second, whether trust developed over a period can equate to "loyalty" and third whether a less than scrupulous Agent can exploit the individual nature of trust to its advantage.

show abstract

“…The vast majority of the work in this eld has focused on making agents more knowledgeable and able. This has been achieved in several ways: by giving the deliberative a g e n t a deeper knowledge base and ability to reason about data 21], giving it the ability to plan actions 26], negotiate with other agents 20], or change its strategies in response to actions of other agents 23,8]. At the opposite end of the spectrum lie agent-based systems that demonstrate complex group behavior, but whose individual elements are very simple.…”

Section: Individual Vs Emergent Complexitymentioning

confidence: 99%

Design and Mathematical Analysis of Agent-Based Systems

Lerman

2001

Formal Approaches to Agent-Based Systems

View full text Add to dashboard Cite

Abstract. Agent-based systems that are composed of simple locally interacting agents but which demonstrate complex group behavior o er several advantages over traditional multi-agent systems. A well-designed complex agent-based systems is an e cient, robust, adaptive and stable system. It has very low communication and computational requirements, meaning that there are virtually no constraints on the system size. The simplicity o f a g e n t i n teractions also makes it amenable to quantitative mathematical analysis. In addition to o ering predictive p o wer, mathematical analysis enables the system designer to optimize system performance. To date, there have been relatively few implementations of complex agent-based systems, mainly because of the di culty of determining what simple agent strategies will lead to desirable collective behavior in a large system. We claim that there exists a set of primitive agent strategies, similar to the basis behaviors in behavior-based robotics, from which complex group behavior can be designed. Moreover, these simple primitive strategies naturally lend themselves to mathematical description, making a quantitative study of agent-based systems possible. We present a case study of coalition formation to show that two simple behaviors, dispersion and aggregation, can lead to coalition formation in a multi-agent system under some conditions. We use this system to illustrate the process by which a mathematical description of the agent-based system is created and analyzed, and discuss the insights the analysis provides for designing coalition forming agents.

show abstract

Adaptive Load Balancing: A Study in Multi-Agent Learning

Cited by 133 publications

References 31 publications

Theory of Collective Intelligence

Theory of Collective Intelligence

Experiments in Building Experiential Trust in a Society of Objective-Trust Based Agents

Design and Mathematical Analysis of Agent-Based Systems

Contact Info

Product

Resources

About