A dual-memory architecture for reinforcement learning on neuromorphic platforms

Abstract

Reinforcement learning (RL) is a foundation of learning in biological systems and provides a framework to address numerous challenges with real-world artificial intelligence applications. Efficient implementations of RL techniques could allow for agents deployed in edge-use cases to gain novel abilities, such as improved navigation, understanding complex situations and critical decision making. Toward this goal, we describe a flexible architecture to carry out RL on neuromorphic platforms. This architecture was implemented using an Intel neuromorphic processor and demonstrated solving a variety of tasks using spiking dynamics. Our study proposes a usable solution for real-world RL applications and demonstrates applicability of the neuromorphic platforms for RL problems.

Experience Replay Optimization via ESMM for Stable Deep Reinforcement Learning

“…Equally, prioritized stochastic memory management (PSMM) [20], combined experience replay (CER) [37], selective experience replay (SER) [17], and episodic memory control (EMC) [40] use experience retention www.ijacsa.thesai.org strategies (memory management strategies). In contrast, some replay strategies focus on the structure of the replay memory instead of the content [35], [42], [43]. ERO has proven superior among prioritized selection algorithms, owing to its easy adaptation and generalization to multiple environments [23].…”

Experience Replay Optimisation via ATSC and TSC for Performance Stability in Deep RL

“…Just as multiple neural networks are used in some RL algorithms to enhance training stability [8,11,15,26,30,31], recent works in ER are exploring the use of a dual-memory architecture. The dualism may come in the form of long and short memory [32] or main and cache [33]. It may also be differentiated based on the sources of the replay data or the ratio of selection from the dual memory.…”

“…It may also be differentiated based on the sources of the replay data or the ratio of selection from the dual memory. Olin-Ammentorp et al [32] rely on the complementary learning system of the human brain (interaction between the cortical and hippocampal networks) to design a dual memory (short-term and long-term) replay architecture. However, their design was implemented in a discrete state-action space.…”