Adaptive policy gradient in multiagent learning

Banerjee, Bikramjit; Peng, Jing

doi:10.1145/860685.860686

Cited by 15 publications

(24 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model of repeated games has been adopted by different researchers to represent complex multiagent interaction schemas. Computer scientists, however, mainly focused on the stationary solutions of repeated games, by only using its repetitive property as a mean to implement an iterative algorithm searching for a stationary equilibrium solution (Bowling & Veloso, 2002;Banerjee & Peng, 2003;Conitzer & Sandholm, 2007). While stationary equilibria are the appropriate solutions in the repeated games, their corresponding set of players' payoffs is typically very restricted.…”

Section: Resultsmentioning

confidence: 99%

“…As a matter of fact, the fundamental repetitive property of repeated games has been typically reduced to a way to give to an adaptive (or learning) algorithm a sufficient time to converge (often, jointly with another learning agent) to a fixed behavior (Bowling & Veloso, 2002;Banerjee & Peng, 2003;Conitzer & Sandholm, 2007;Burkov & Chaib-draa, 2009). Thus, the repetitive nature of the game has only been considered as a permissive property, that is, a property that permits implementing an algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Repeated games for multiagent systems: a survey

Burkov

Chaib-draa

2013

The Knowledge Engineering Review

View full text Add to dashboard Cite

Repeated games are an important mathematical formalism to model and study long-term economic interactions between multiple self-interested parties (individuals or groups of individuals). They open attractive perspectives in modeling long-term multiagent interactions. This overview paper discusses the most important results that actually exist for repeated games. These results arise from both economics and computer science. Contrary to a number of existing surveys of repeated games, most of which originated from the economic research community, we are first to pay a special attention to a number of important distinctive features proper to artificial agents. More precisely, artificial agents, as opposed to the human agents mainly aimed by the economic research, are usually bounded whether in terms of memory or performance. Therefore, their decisions have to be based on the strategies defined using finite representations. Furthermore, these strategies have to be efficiently computed or approximated using a limited computational resource usually available to artificial agents.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Repeated games for multiagent systems: a survey

Burkov

Chaib-draa

2013

The Knowledge Engineering Review

View full text Add to dashboard Cite

show abstract

“…In an effort to overcome the drawback of declining solution quality with large problem instances, we have integrated the variable step-size hill-climbing algorithm PDWoLF [9] into several legacy ACO algorithms. The following reviews off policy-based learning, variable-step hill-climbing, and PDWoLF.…”

Section: Policy Dynamics Win or Learn Fast (Pdwolf)mentioning

confidence: 99%

“…In the Policy Dynamics Win or Learn Fast Policy HillClimbing (PDWoLF-PHC) algorithm [9] Banerjee and Peng replaced the notion of average policy contained in equation (5) with one that uses the gradient of the policy. Otherwise, WoLF-PHC and PDWoLF-PHC are identical.…”

Section: Policy Dynamics Win or Learn Fast (Pdwolf)mentioning

confidence: 99%

See 1 more Smart Citation

WoLF Ant

Peterson

Mayer

Cousin

2011

2011 IEEE Congress of Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Ant colony optimization (ACO) algorithms can generate quality solutions to combinatorial optimization problems. However, like many stochastic algorithms, the quality of solutions worsen as problem sizes grow. In an effort to increase performance, we added the variable step size off-policy hill-climbing algorithm called PDWoLF (Policy Dynamics Win or Learn Fast) to several ant colony algorithms: Ant System, Ant Colony System, Elitist-Ant System, Rank-based Ant System, and Max-Min Ant System. Easily integrated into each ACO algorithm, the PDWoLF component maintains a set of policies separate from the ant colony's pheromone. Similar to pheromone but with different update rules, the PDWoLF policies provide a second estimation of solution quality and guide the construction of solutions. Experiments on large traveling salesman problems (TSPs) show that incorporating PDWoLF with the aforementioned ACO algorithms that do not make use of local optimizations produces shorter tours than the ACO algorithms alone.

show abstract

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Zhang

Hao

et al. 2019

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

In multiagent environments, the capability of learning is important for an agent to behave appropriately in face of unknown opponents and dynamic environment. From the system designer's perspective, it is desirable if the agents can learn to coordinate towards socially optimal outcomes, while also avoiding being exploited by selfish opponents. To this end, we propose a novel gradient ascent based algorithm (SA-IGA) which augments the basic gradient-ascent algorithm by incorporating social awareness into the policy update process. We theoretically analyze the learning dynamics of SA-IGA using dynamical system theory and SA-IGA is shown to have linear dynamics for a wide range of games including symmetric games. The learning dynamics of two representative games (the prisoner's dilemma game and the coordination game) are analyzed in details. Based on the idea of SA-IGA, we further propose a practical multiagent learning algorithm, called SA-PGA, based on Q-learning update rule. Simulation results show that SA-PGA agent can achieve higher social welfare than previous social-optimality oriented Conditional Joint Action Learner (CJAL) and also is robust against individually rational opponents by reaching Nash equilibrium solutions.

show abstract

Adaptive policy gradient in multiagent learning

Cited by 15 publications

References 0 publications

Repeated games for multiagent systems: a survey

Repeated games for multiagent systems: a survey

WoLF Ant

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Contact Info

Product

Resources

About