A quantitative analysis of memory requirement and generalization performance for robotic tasks

Kim, DaeEun

doi:10.1145/1276958.1277015

Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation 2007

DOI: 10.1145/1276958.1277015

|View full text |Cite

A quantitative analysis of memory requirement and generalization performance for robotic tasks

DaeEun Kim

Abstract: In autonomous agent systems, memory is an important element to handle agent behaviors appropriately. We present the analysis of memory requirements for robotic tasks including wall following and corridor following. The robotic tasks are simulated with sensor modeling and motor actions in noisy environments. In this paper, control structures are based on finite state machines for memory-based controllers, and we use the evolutionary multiobjective optimization approach with two objectives, behavior performance … Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Publication Types

Select...

Other1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Reward-driven Training of Random Boolean Network Reservoirs for Model-Free Environments

Gargesa¹

2000

View full text Add to dashboard Cite

Reservoir Computing (RC) is an emerging machine learning paradigm where a fixed kernel, built from a randomly connected "reservoir" with sufficiently rich dynamics, is capable of expanding the problem space in a non-linear fashion to a higher dimensional feature space. These features can then be interpreted by a linear readout layer that is trained by a gradient descent method. In comparison to traditional neural networks, only the output layer needs to be trained, which leads to a significant computational advantage. In addition, the short term memory of the reservoir dynamics has the ability to transform a complex temporal input state space to a simple non-temporal representation. Adaptive real-time systems are multi-stage decision problems that can be used to train an agent to achieve a preset goal by performing an optimal action at each timestep. In such problems, the agent learns through continuous interactions with its environment. Conventional techniques to solving such problems become computationally expensive or may not converge if the state-space being considered is large, partially observable, or if short term memory is required in optimal decision making. The objective of this thesis is to use reservoir computers to solve such goaldriven tasks, where no error signal can be readily calculated to apply gradient descent methodologies. To address this challenge, we propose a novel reinforcement learning approach in combination with reservoir computers built from simple Boolean components. Such reservoirs are of interest because they have the potential to be fabricated by self-assembly techniques. We evaluate the performance of our approach in both Markovian and non-Markovian environments. We compare the performance of an agent trained through traditional Q-Learning. We find that i the reservoir-based agent performs successfully in these problem contexts and even performs marginally better than Q-Learning agents in certain cases. Our proposed approach allows to retain the advantage of traditional parameterized dynamic systems in successfully modeling embedded state-space representations while eliminating the complexity involved in training traditional neural networks. To the best of our knowledge, our method of training a reservoir readout layer through an on-policy boot-strapping approach is unique in the field of random Boolean network reservoirs.

show abstract

Reward-driven Training of Random Boolean Network Reservoirs for Model-Free Environments

Gargesa¹

2000

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

A quantitative analysis of memory requirement and generalization performance for robotic tasks

Cited by 1 publication

References 17 publications

Reward-driven Training of Random Boolean Network Reservoirs for Model-Free Environments

Reward-driven Training of Random Boolean Network Reservoirs for Model-Free Environments

Contact Info

Product

Resources

About