Dynamic analysis of multiagent
            <i>Q</i>
            -learning with ε-greedy exploration

Gomes, Eduardo Rodrigues; Kowalczyk, Ryszard

doi:10.1145/1553374.1553422

Cited by 55 publications

(17 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other authors [Babes et al, 2009;Gomes and Kowalczyk, 2009; independently arrived at the same expected change of Q-values. However, these sources explicitly consider ǫ-greedy as the policy generation function, which maps Q-values to a few discrete policies and thus does not allow the policy space of the process to be described in a self-consistent way.…”

Section: Discrepancy Between Q-learning and Its Idealized Modelsupporting

confidence: 56%

Learning against learning : evolutionary dynamics of reinforcement learning algorithms in strategic interactions

Kaisers¹

View full text Add to dashboard Cite

People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User Agreement:

show abstract

Section: Discrepancy Between Q-learning and Its Idealized Modelsupporting

confidence: 56%

Learning against learning : evolutionary dynamics of reinforcement learning algorithms in strategic interactions

Kaisers¹

View full text Add to dashboard Cite

show abstract

“…The experimental parameters are as follows: α = 0.05, γ = 0.9, and ε = 0.85. In each episode from line 11 to line 31, the model chooses an action a ( i , j ) given current state s based on ε -greedy [ 21 ] and receives a new state s' (lines 12∼13). There are two conditions in which the episode will end.…”

Section: Methodology and Learning Designmentioning

confidence: 99%

An Empirical Investigation of Transfer Effects for Reinforcement Learning

Jwo

Lin

Lee

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Previous studies have shown that training a reinforcement model for the sorting problem takes very long time, even for small sets of data. To study whether transfer learning could improve the training process of reinforcement learning, we employ Q-learning as the base of the reinforcement learning algorithm, apply the sorting problem as a case study, and assess the performance from two aspects, the time expense and the brain capacity. We compare the total number of training steps between nontransfer and transfer methods to study the efficiencies and evaluate their differences in brain capacity (i.e., the percentage of the updated Q-values in the Q-table). According to our experimental results, the difference in the total number of training steps will become smaller when the size of the numbers to be sorted increases. Our results also show that the brain capacities of transfer and nontransfer reinforcement learning will be similar when they both reach a similar training level.

show abstract

“…An important aspect of understanding the behavior of a multiagent learning algorithm is theoretically modeling and analyzing its underlying dynamics [6,20,25].…”

Section: Theoretical Modeling and Analysis Of Sa-igamentioning

confidence: 99%

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Zhang

Hao

et al. 2019

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

In multiagent environments, the capability of learning is important for an agent to behave appropriately in face of unknown opponents and dynamic environment. From the system designer's perspective, it is desirable if the agents can learn to coordinate towards socially optimal outcomes, while also avoiding being exploited by selfish opponents. To this end, we propose a novel gradient ascent based algorithm (SA-IGA) which augments the basic gradient-ascent algorithm by incorporating social awareness into the policy update process. We theoretically analyze the learning dynamics of SA-IGA using dynamical system theory and SA-IGA is shown to have linear dynamics for a wide range of games including symmetric games. The learning dynamics of two representative games (the prisoner's dilemma game and the coordination game) are analyzed in details. Based on the idea of SA-IGA, we further propose a practical multiagent learning algorithm, called SA-PGA, based on Q-learning update rule. Simulation results show that SA-PGA agent can achieve higher social welfare than previous social-optimality oriented Conditional Joint Action Learner (CJAL) and also is robust against individually rational opponents by reaching Nash equilibrium solutions.

show abstract

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

Cited by 55 publications

References 16 publications

Learning against learning : evolutionary dynamics of reinforcement learning algorithms in strategic interactions

Learning against learning : evolutionary dynamics of reinforcement learning algorithms in strategic interactions

An Empirical Investigation of Transfer Effects for Reinforcement Learning

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

Contact Info

Product

Resources

About