Performance Evaluation of an Evolutionary Method for RoboCup Soccer Strategies

Nakashima, Tomoharu; Takatani, M.; Udo, Masayo; Ishibuchi, Hisao; Nii, Manabu

doi:10.1007/11780519_61

Cited by 14 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, there is team learning, where a single learning algorithm is used to optimize the behaviors of an entire team. Some of this work has involved evolutionary computation methods to develop joint team behaviors (such as [15,10]); reinforcement learning papers have instead usually developed a single homogeneous behavior (for example [7,22]). In contrast the concurrent learning literature, where separate learners are applied per-agent, has largely applied multiagent reinforcement learning (such as [12,21]).…”

Section: Machine Learning At Robocupmentioning

confidence: 99%

Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

Freelan

Wicke

Sullivan

et al. 2015

RoboCup 2014: Robot World Cup XVIII

View full text Add to dashboard Cite

Abstract. We describe our previous and current efforts towards achieving an unusual personal RoboCup goal: to train a full team of robots directly through demonstration, on the field of play at the RoboCup venue, how to collaboratively play soccer, and then use this trained team in the competition itself. Using our method, HiTAB, we can train teams of collaborative agents via demonstration to perform nontrivial joint behaviors in the form of hierarchical finite-state automata. We discuss HiTAB, our previous efforts in using it in RoboCup 2011 and 2012, recent experimental work, and our current efforts for 2014, then suggest a new RoboCup Technical Challenge problem in learning from demonstration.Imagine that you are at an unfamiliar disaster site with a team of robots, and are faced with a previously unseen task for them to do. The robots have only rudimentary but useful utility behaviors implemented. You are not a programmer. Without coding them, you have only a few hours to get your robots doing useful collaborative work in this new environment. How would you do this?Our interest lies in rapid, real-time multi-robot training from demonstration. Here a single human trainer teaches a team of robots, via teleoperation, how to collectively perform tasks in previously unforeseen environments. This is difficult for two reasons. First, nontrivial behaviors can present a high-dimensional space to learn, yet one can only provide a few samples, as online training samples are costly to collect. This is a worst case for the so-called "curse of dimensionality". Second, when training multiple interactive robots, even if you can quantify the emergent macro-level group behavior you wish to achieve, in order to do learning, each agent needs to know the micro-level behavior he is being asked to do. One may have a micro→macro function (a simulator), but it is unlikely that one has the inverse macro→micro function, resulting in what we call the "multiagent inverse problem". These two challenges mean that real-time multi-robot learning from demonstration has proven very difficult and has a very sparse literature.Over the past several years we have participated in the Kid-Size Humanoid League with a single objective: to successfully do a personal RoboCup-style technical challenge of our own invention, independent of those offered at RoboCup: can we train multiple generic robots, through demonstration on the field, how to play collaborative soccer at RoboCup solely within the preparatory time prior to the competition itself ?

show abstract

Section: Machine Learning At Robocupmentioning

confidence: 99%

Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

Freelan

Wicke

Sullivan

et al. 2015

RoboCup 2014: Robot World Cup XVIII

View full text Add to dashboard Cite

show abstract

“…Miconi [14] and Nakashima et al [53] used 1-point and 2-point crossover to recombine the teams' genotypes. The npoint crossover might be considered as a competitive way to implement RAS.…”

Section: Appendix a 1-point And 2-point Crossovermentioning

confidence: 99%

“…The family of uniform crossovers was introduced by Syswerda [50] and analytically studied by Eshelman et al [51] and De Jong and Spears [52], but not in the context of team evolution. In addition, Miconi [14] and Nakashima et al [53] used a simple 1-point or 2-point crossover to recombine the teams' genotypes. This approach leans itself to the category of restricted crossovers.…”

Section: Introductionmentioning

confidence: 99%

Evolving Team Compositions by Agent Swapping

Lichocki

Wischmann

Keller

et al. 2013

IEEE Trans. Evol. Computat.

View full text Add to dashboard Cite

Abstract-Optimizing collective behavior in multiagent systems requires algorithms to find not only appropriate individual behaviors but also a suitable composition of agents within a team. Over the last two decades, evolutionary methods have been shown to be a promising approach for the design of agents and their compositions into teams. The choice of a crossover operator that facilitates the evolution of optimal team composition is recognized to be crucial, but so far it has never been thoroughly quantified. Here we highlight the limitations of two different crossover operators that exchange entire agents between teams: restricted agent swapping that exchanges only corresponding agents between teams and free agent swapping that allows an arbitrary exchange of agents. Our results show that restricted agent swapping suffers from premature convergence, whereas free agent swapping entails insufficient convergence. Consequently, in both cases the exploration and exploitation aspects of the evolutionary algorithm are not well balanced resulting in the evolution of suboptimal team compositions. To overcome this problem we propose to combine the two methods. Our approach first applies free agent swapping to explore the search space and then restricted agent swapping to exploit it. This mixed approach turns out to be a much more efficient strategy for the evolution of team compositions compared to either strategy alone. Our results suggest that such a mixed agent swapping algorithm should always be preferred whenever the optimal composition of individuals in a multiagent system is unknown.

show abstract

“…Research groups have applied a variety of different machine learning methods to many aspects of autonomously soccer playing multirobot systems. Examples include evolutionary algorithms for gait optimization (Chernova and Veloso 2004; Röfer et al 2004) or optimization of team tactics (Nakashima et al 2005), unsupervised and supervised learning in computer vision tasks (Kaufmann et al 2004;Li et al 2003;Treptow and Zell 2004) and lower level control tasks (Oubbati et al 2005). RL methods have been used to learn cooperative behaviors in the simulation league (Ma et al 2008) as well as for real robots (Asada et al 1999) and to learn walking patterns on humanoid robots (Ogino et al 2004).…”

Section: Related Workmentioning

confidence: 99%

Reinforcement learning for robot soccer

et al. 2009

View full text Add to dashboard Cite

Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning framework, particularly tailored to the use of multilayer perceptrons to approximate value functions over continuous state spaces. The batch learning framework is successfully used to learn crucial skills in our soccer-playing robots participating in the RoboCup competitions. This is demonstrated on three different case studies.

show abstract

Performance Evaluation of an Evolutionary Method for RoboCup Soccer Strategies

Cited by 14 publications

References 4 publications

Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

Evolving Team Compositions by Agent Swapping

Reinforcement learning for robot soccer

Contact Info

Product

Resources

About