Multi-agent Behavior-Based Policy Transfer

Didi, Sabre; Nitschke, Geoff

doi:10.1007/978-3-319-31153-1_13

Cited by 16 publications

(12 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, all hybrid evolved behaviors were significantly more effective (and efficiently evolved) compared to those evolved by other evolutionary search variants. This result was supported by related work that similarly demonstrated the benefits of combining behavioral diversity maintenance and objective-based evolutionary search (Gomes et al, 2013;Shorten and Nitschke, 2015;Didi and Nitschke, 2016b) to mitigate the bootstrap problem (Gomez and Miikkulainen, 1997) for complex tasks and enable the evolution of high-quality solutions. The third insight was that the high-quality behaviors evolved by the hybrid evolutionary search method encoded relatively simple neural controllers.…”

Section: Resultssupporting

confidence: 52%

“…Previous work indicated that keep-away task complexity increases with the number of taker and keeper agents (Whiteson et al, 2005;Stone et al, 2006a;Didi and Nitschke, 2016b). Complexity refers to task difficulty and thus the level of sophistication required by evolved behaviors to solve the task.…”

Section: Keep-away Task Complexitymentioning

confidence: 99%

“…For example, as the number of agents changes between keep-away tasks, the task complexity also changes (Section 3.6), though connectivity patterns encoded in evolved CPPNs have been demonstrated as readily transferable across different tasks . In preliminary parameter tuning experiments done in previous research (Didi and Nitschke, 2016b), several approaches for collective behavior policy transfer were tested, though the following approach was found to be the most effective for all HyperNEAT variants and all keep-away tasks tested in this study. Specifically, the entire evolved population was transferred from the source task (at the final generation of neuroevolution) and set as the initial population for keep-away behavior evolution in the target task.…”

Section: Collective Behavior (Policy) Transfermentioning

confidence: 99%

“…Given previous work elucidating the efficacy of hybrid evolutionary search (Gomes and Christensen, 2013b;Gomes et al, 2013Gomes et al, , 2015 and policy transfer (Taylor et al, 2006;Verbancsics and Stanley, 2010;Didi and Nitschke, 2016b) in collective behavior tasks the following hypothesis forms the research focus of this study.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Nitschke

Didi

2017

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

This study evaluates various evolutionary search methods to direct neural controller evolution in company with policy (behavior) transfer across increasingly complex collective robotic (RoboCup keep-away) tasks. Robot behaviors are first evolved in a source task and then transferred for further evolution to more complex target tasks. Evolutionary search methods tested include objective-based search (fitness function), behavioral and genotypic diversity maintenance, and hybrids of such diversity maintenance and objective-based search. Evolved behavior quality is evaluated according to effectiveness and efficiency. Effectiveness is the average task performance of transferred and evolved behaviors, where task performance is the average time the ball is controlled by a keeper team. Efficiency is the average number of generations taken for the fittest evolved behaviors to reach a minimum task performance threshold given policy transfer. Results indicate that policy transfer coupled with hybridized evolution (behavioral diversity maintenance and objective-based search) addresses the bootstrapping problem for increasingly complex keep-away tasks. That is, this hybrid method (coupled with policy transfer) evolves behaviors that could not otherwise be evolved. Also, this hybrid evolutionary search was demonstrated as consistently evolving topologically simple neural controllers that elicited high-quality behaviors.

show abstract

Section: Resultssupporting

confidence: 52%

Section: Keep-away Task Complexitymentioning

confidence: 99%

Section: Collective Behavior (Policy) Transfermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Nitschke

Didi

2017

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Kelly and Heywood [23] introduced a symbiotic framework that hierarchically constructs a policy tree to transfer tasks under genetic programming. Didi and Nitschke [24] used the neuro-evolution method to evolve policy from the source task to the target task. The ADAAPT (A Deep Architecture for Adaptive Policy Transfer) method proposed in [25] combines the weighted learned policies for knowledge transfer, and updates an attention work during the reinforcement learning to adjust the policy weights.…”

Section: Literature Reviewmentioning

confidence: 99%

Reusing Source Task Knowledge via Transfer Approximator in Reinforcement Transfer Learning

2018

View full text Add to dashboard Cite

Transfer Learning (TL) has received a great deal of attention because of its ability to speed up Reinforcement Learning (RL) by reusing learned knowledge from other tasks. This paper proposes a new transfer learning framework, referred to as Transfer Learning via Artificial Neural Network Approximator (TL-ANNA). It builds an Artificial Neural Network (ANN) transfer approximator to transfer the related knowledge from the source task into the target task and reuses the transferred knowledge with a Probabilistic Policy Reuse (PPR) scheme. Specifically, the transfer approximator maps the state of the target task symmetrically to states of the source task with a certain mapping rule, and activates the related knowledge (components of the action-value function) of the source task as the input of the ANNs; it then predicts the quality of the actions in the target task with the ANNs. The target learner uses the PPR scheme to bias the RL with the suggested action from the transfer approximator. In this way, the transfer approximator builds a symmetric knowledge path between the target task and the source task. In addition, two mapping rules for the transfer approximator are designed, namely, Full Mapping Rule and Group Mapping Rule. Experiments performed on the RoboCup soccer Keepaway task verified that the proposed transfer learning methods outperform two other transfer learning methods in both jumpstart and time to threshold metrics and are more robust to the quality of source knowledge. In addition, the TL-ANNA with the group mapping rule exhibits slightly worse performance than the one with the full mapping rule, but with less computation and space cost when appropriate grouping method is used.

show abstract

Hybridizing novelty search for transfer learning

Didi

Nitschke

2016

2016 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

Multi-agent Behavior-Based Policy Transfer

Cited by 16 publications

References 27 publications

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Reusing Source Task Knowledge via Transfer Approximator in Reinforcement Transfer Learning

Hybridizing novelty search for transfer learning

Contact Info

Product

Resources

About