A Cognitive Architecture Based on a Learning Classifier System with Spiking Classifiers

Abstract

Learning Classifier Systems (LCS) are population-based reinforcement learners that were originally designed to model various cognitive phenomena. This paper presents an explicitly cognitive LCS by using spiking neural networks as classifiers, providing each classifier with a measure of temporal dynamism. We employ a constructivist model of growth of both neurons and synaptic connections, which permits a Genetic Algorithm (GA) to automatically evolve sufficiently-complex neural structures. The spiking classifiers are coupled with a temporally-sensitive reinforcement learning algorithm, which allows the system to perform temporal state decomposition by appropriately rewarding "macro-actions", created by chaining together multiple atomic actions. The combination of temporal reinforcement learning and neural information processing is shown to outperform benchmark neural classifier systems, and successfully solve a robotic navigation task.

Accuracy-Based Learning Classifier Systems for Multistep Reinforcement Learning: A Fuzzy Logic Approach to Handling Continuous Inputs and Learning Continuous Actions

“…Particularly, the single-step Frog problem is utilized first to examine whether our LFCS can quickly respond to immediate environmental feedback and learn to achieve its goal by performing just one action. Thereafter, our LFCS will be further tested on several multistep learning problems, including the Cart-Pole problem, the Puddle World problem, the Mountain Car problem, and the Robotics problem introduced in [34]. Finally, to demonstrate the practical usefulness of our LFCS, the algorithm will also be utilized to improve the performance of Wireless Body Area Networks (WBANs).…”

“…In [34], Howard et al developed another interesting algorithm that seamlessly merged spiking neural network based classifiers with the temporal classifier system (TCS). While we concentrate on solving Markov Decision Processes (MDPs) through learning, their algorithm was explicitly designed to address semi-MDPs by using temporally persistent actions and a temporal reinforcement learning algorithm.…”

“…While we concentrate on solving Markov Decision Processes (MDPs) through learning, their algorithm was explicitly designed to address semi-MDPs by using temporally persistent actions and a temporal reinforcement learning algorithm. Meanwhile, all learning tasks in [34] require discrete actions. The study of neuro-evolutionary algorithm in [34] is also beyond the scope of our research in this paper.…”

“…Meanwhile, all learning tasks in [34] require discrete actions. The study of neuro-evolutionary algorithm in [34] is also beyond the scope of our research in this paper.…”

“…In particular, we have examined the learning performance on four benchmark learning problems as well as one robotics problem introduced in [34]. For a comparison study, four recently proposed reinforcement learning algorithms have been implemented and tested.…”