Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study

Kalyanakrishnan, Shivaram; Liu, Yaxin; Stone, Peter

doi:10.1007/978-3-540-74024-7_7

Cited by 62 publications

(55 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Around the 900th trial, the learning seems to have converged at 30% success, 70% failure, and 10% draw. Compared to the results of [5] that has around 30% success rate with 30,000 trials, the learning time is drastically improved (30 times quicker). Fig.…”

Section: Resultsmentioning

confidence: 80%

“…As a result, we have much improved the learning time (30 times quicker!) compared to the result of the existing method [5] that has 32% success with communication and 23% without communication at around the 30,000th trial when the learning seems to have converged.…”

Section: Discussionmentioning

confidence: 92%

“…Recently, there have been increasing number of studies on cooperative/competitive behavior acquisition in a multiagent environment by using reinforcement learning methods [2,3,[5][6][7]. In such an environment, the state and action spaces for the learning can be easily exploded since not only objects but also other agents should be involved in the state and action spaces, and therefore the sensor and actuator level descriptions may cause information explosion that disables the learning methods to be applied within practical learning time.…”

Section: Introductionmentioning

confidence: 99%

“…In such an environment, the state and action spaces for the learning can be easily exploded since not only objects but also other agents should be involved in the state and action spaces, and therefore the sensor and actuator level descriptions may cause information explosion that disables the learning methods to be applied within practical learning time. Shivaram et al [5] showed that the learning rate can be accelerated by sharing the learned information in the 4 on 5 game task. However, they need still long learning time since they directly use the sensory information as state value to decide the situation.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cooperative/Competitive Behavior Acquisition Based on State Value Estimation of Others

Noma

Takahashi

Asada

2008

RoboCup 2007: Robot Soccer World Cup XI

View full text Add to dashboard Cite

Abstract. The existing reinforcement learning approaches have been suffering from the curse of dimension problem when they are applied to multiagent dynamic environments. One of the typical examples is a case of RoboCup competitions since other agents and their behaviors easily cause state and action space explosion. This paper presents a method of modular learning in a multiagent environment by which the learning agent can acquire cooperative behaviors with its team mates and competitive ones against its opponents. The key ideas to resolve the issue are as follows. First, a two-layer hierarchical system with multi learning modules is adopted to reduce the size of the sensor and action spaces. The state space of the top layer consists of the state values from the lower level, and the macro actions are used to reduce the size of the physical action space. Second, the state of the other to what extent it is close to its own goal is estimated by observation and used as a state value in the top layer state space to realize the cooperative/competitive behaviors. The method is applied to 4 (defence team) on 5 (offence team) game task, and the learning agent successfully acquired the teamwork plays (pass and shoot) within much shorter learning time (30 times quicker than the earlier work).

show abstract

Section: Resultsmentioning

confidence: 80%

Section: Discussionmentioning

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cooperative/Competitive Behavior Acquisition Based on State Value Estimation of Others

Noma

Takahashi

Asada

2008

RoboCup 2007: Robot Soccer World Cup XI

View full text Add to dashboard Cite

show abstract

“…The reinforcement given to the agents were inspired on the definitions of rewards presented in [3], and are different for the agents. For the goalkeeper, the rewards consists of: ball caught, kicked or driven by goalie = 25; ball with any opponent player = -25; goal scored by the opponent = -100.…”

Section: Experiments In the Robocup 2d Simulation Domainmentioning

confidence: 99%

Heuristic Reinforcement Learning Applied to RoboCup Simulation Agents

Celiberto

Ribeiro

Costa

et al. 2008

RoboCup 2007: Robot Soccer World Cup XI

View full text Add to dashboard Cite

Abstract. This paper describes the design and implementation of robotic agents for the RoboCup Simulation 2D category that learns using a recently proposed Heuristic Reinforcement Learning algorithm, the Heuristically Accelerated Q-Learning (HAQL). This algorithm allows the use of heuristics to speed up the well-known Reinforcement Learning algorithm Q-Learning. A heuristic function that influences the choice of the actions characterizes the HAQL algorithm. A set of empirical evaluations was conducted in the RoboCup 2D Simulator, and experimental results show that even very simple heuristics enhances significantly the performance of the agents.

show abstract

Reinforcement Learning for Scheduling of Maintenance

Knowles

Baglee

Wermter

2010

Research and Development in Intelligent Systems XXVII

View full text Add to dashboard Cite

Improving maintenance scheduling has become an area of crucial importance in recent years. Condition-based maintenance (CBM) has started to move away from scheduled maintenance by providing an indication of the likelihood of failure. Improving the timing of maintenance based on this information to maintain high reliability without resorting to over-maintenance remains, however, a problem. In this paper we propose Reinforcement Learning (RL), to improve long term reward for a multistage decision based on feedback given either during or at the end of a sequence of actions, as a potential solution to this problem. Several indicative scenarios are presented and simulated experiments illustrate the performance of RL in this application.

show abstract

Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study

Cited by 62 publications

References 5 publications

Cooperative/Competitive Behavior Acquisition Based on State Value Estimation of Others

Cooperative/Competitive Behavior Acquisition Based on State Value Estimation of Others

Heuristic Reinforcement Learning Applied to RoboCup Simulation Agents

Reinforcement Learning for Scheduling of Maintenance

Contact Info

Product

Resources

About