Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling

Ramon, Jan; Driessens, Kurt; Croonenborghs, Tom

doi:10.1007/978-3-540-74958-5_70

Cited by 47 publications

(22 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ramon et al (2007) extend the TG algorithm with tree restructuring operations based on ideas of the incremental tree induction (ITI) algorithm by Utgoff (1997) and concepts from theory revision in ILP. New operations in this new algorithm TGR include the pruning of a leaf or a whole subtree, and the revision of an internal node.…”

Section: Logical Abstractionsmentioning

confidence: 99%

Logic of Adaptive Behavior

Otterlo¹

View full text Add to dashboard Cite

PrefaceOne the central themes of this book is the interplay between representation and behavior. Representation is about the way we look at our world, about how we see the things around us, and about how we think about and evaluate the concepts and knowledge that shape our surroundings.Behavior is about what we do, about our actions in the world or the mental actions in our head, and about the consequences of a wide variety of activities we engage in every day. Our representation of the world determines our behavior, but at the same time, our behavior influences what we see, and with that, it changes our representation of the world. This strange, mutually modifying loop is a fascinating phenomenon and it is the core object of study in both psychology and artificial intelligence. Indeed, much of the material in this book is about formal models and algorithms that characterize how computer-based, intelligent systems can represent their mathematical worlds and how they can behave rationally in such worlds. However, the pattern of this loop is much more general, and can be observed almost anywhere. For example, every paper I read changes my representation of my own scientific world, but at the same time, my research activities determine my choices for further study.Out of the many possible things that may be present in your representation of your world, and that may be part of your behavioral pattern, are people. Most of our representations of our world, our thinking, our talking, and our behavioral patterns, involve people. Especially in the path leading to this book, my representation of the world, and my course of action, have been influenced by so many people. For starters, in the writing of this book I have been influenced by many great scientists. There are many of them who I know personally and with whom I had the privilege to engage in interesting interactions, but there are many others that have influenced me without them knowing, for example when I attended their talks or read their books and papers. At this place, I want to express my gratitude to a number of specific people that have influenced me, helped me, supported me and made my life so much more interesting and enjoyable.First of all, I would like to thank my promotor Anton Nijholt for giving me the opportunity to do a PhD in his research group, and for the complete freedom and support he has given me to pursue my own interests, even when my path led me to new topics, to Utrecht and even to Freiburg. I thank Mannes Poel for all his support, his open-door policy that made it possible to just drop by with a coffee and start a discussion on any topic that was on my mind at those specific moments, for all the fun lunch walks, the stimulating educational activities we did together and all the other things that made our interactions fun and interesting.Many thanks go to my promotor John-Jules Meyer. His endless kindness, support and expertise have had great influence on my journey through my PhD period. I vividly remember our productive discussions, alwa...

show abstract

Section: Logical Abstractionsmentioning

confidence: 99%

Logic of Adaptive Behavior

Otterlo¹

View full text Add to dashboard Cite

show abstract

“…Policy transfer used in company with various RL methods has been applied to boost solution quality in various single-agent tasks including pole-balancing (Ammar et al, 2012), gameplaying (Ramon et al, 2007), robot navigation, as well as multiagent tasks including predator-prey (Boutsioukis et al, 2012). For such single and multiagent tasks, policy transfer is typically done within the same task domain for varying task complexity (Torrey and Shavlik, 2009) and less frequently between different task domains (Bou-Ammar et al, 2015).…”

Section: Evolutionary Policy Transfermentioning

confidence: 99%

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Nitschke

Didi

2017

Front. Robot. AI

View full text Add to dashboard Cite

This study evaluates various evolutionary search methods to direct neural controller evolution in company with policy (behavior) transfer across increasingly complex collective robotic (RoboCup keep-away) tasks. Robot behaviors are first evolved in a source task and then transferred for further evolution to more complex target tasks. Evolutionary search methods tested include objective-based search (fitness function), behavioral and genotypic diversity maintenance, and hybrids of such diversity maintenance and objective-based search. Evolved behavior quality is evaluated according to effectiveness and efficiency. Effectiveness is the average task performance of transferred and evolved behaviors, where task performance is the average time the ball is controlled by a keeper team. Efficiency is the average number of generations taken for the fittest evolved behaviors to reach a minimum task performance threshold given policy transfer. Results indicate that policy transfer coupled with hybridized evolution (behavioral diversity maintenance and objective-based search) addresses the bootstrapping problem for increasingly complex keep-away tasks. That is, this hybrid method (coupled with policy transfer) evolves behaviors that could not otherwise be evolved. Also, this hybrid evolutionary search was demonstrated as consistently evolving topologically simple neural controllers that elicited high-quality behaviors.

show abstract

“…The result is a new covering algorithm that utilises "Transfer Knowledge" approach to fill in missing attribute values specifically the target attribute in the training dataset before mining kicks in. The "Transfer Knowledge" is basically building up a knowledge base based on learning curves via agents from different environments and from previous learning experience and then using this knowledge base to fill in incomplete data examples (Ramon et al, 2007). Experimental results against eight datasets revealed that the improved covering algorithm consistently produced competitive classifiers when compared with other RI algorithms such as RULES-IS and PRISM.…”

Section: Prism Consmentioning

confidence: 99%