Exploiting Memristors for Neuromorphic Reinforcement Learning

Shi, Cong; Lu, Junqing; Wang, Ying; Li, Ping; Mao, Tian

doi:10.1109/aicas51828.2021.9458542

Cited by 8 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting plasticity rule models how the synaptic weight is modified, considering only the spiking activity. According to [10][11][12], a global reward signal R is introduced to model neuromodulatory signals. Setting R ∈ [−1, 1], Equation ( 4) is changed then to: ) shows the role of the reward signal R = −1, inverting the role of presynaptic and postsynaptic spikes.…”

Section: R-stdp Learning Rulementioning

confidence: 99%

Building an Analog Circuit Synapse for Deep Learning Neuromorphic Processing

Juarez-Lora,

Ponce-Ponce,

Sossa-Azuela

et al. 2024

Preprint

View full text Add to dashboard Cite

In this article, we propose a circuit to imitate the behavior of a Reward-Modulated Spike-Timing-Dependent Plasticity synapse. When two neurons in adjacent layers produce spikes, each spike modifies the thickness of the common synapse. As a result, the synapse’s ability to conduct impulses is controlled, leading to an unsupervised learning rule. By introducing a reward signal, reinforcement learning is enabled by redirecting the growth and shrinkage of synapses based on signal feedback from the environment. The proposed synapse manages the convolution of the emitted spike signals to promote either the strengthening or weakening of the synapse, which is represented as the resistance value of a memristor device. As memristors have a conductance range that may differ from the available current input range of typical CMOS neuron designs, the synapse circuit can be adjusted to regulate the spike’s amplitude current to comply with the neuron. The circuit described in this work allows for the implementation of fully interconnected layers of neuron analog circuits. This is achieved by having each synapse reconform the spike signal, thus removing the burden of providing enough power from the neurons to each memristor. The synapse circuit was tested using a CMOS analog neuron described in the literature. Additionally, the article provides insight into how to properly describe the hysteresis behavior of the memristor in Verilog-A code. The testing and learning capabilities of the synapse circuit are demonstrated in simulation using the Skywater-130nm process. The article’s main goal is to provide the basic building blocks for Deep Neural Neural Networks relying on spiking neurons and memristors as the basic processing elements to handle spike generation, propagation, and synaptic plasticity.

show abstract

Section: R-stdp Learning Rulementioning

confidence: 99%

Building an Analog Circuit Synapse for Deep Learning Neuromorphic Processing

Juarez-Lora,

Ponce-Ponce,

Sossa-Azuela

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Supervised circuit Application Energy Area CMT [51] Not required Image pattern(X/O) recognition task 32 pJ 100 μm 2 4T1M [52] Not required Cart-pole task N/A N/A of spikes corresponds perfectly to the occurrence of events. In this area, various types of learning rules have been explored to develop event-driven computing and applications.…”

Section: Featuresmentioning

confidence: 99%

Complementary memtransistors for neuromorphic computing: How, what and why

Chen,

Zhou,

Xiong

et al. 2024

J. Semicond.

View full text Add to dashboard Cite

Memtransistors in which the source−drain channel conductance can be nonvolatilely manipulated through the gate signals have emerged as promising components for implementing neuromorphic computing. On the other side, it is known that the complementary metal-oxide-semiconductor (CMOS) field effect transistors have played the fundamental role in the modern integrated circuit technology. Therefore, will complementary memtransistors (CMT) also play such a role in the future neuromorphic circuits and chips? In this review, various types of materials and physical mechanisms for constructing CMT (how) are inspected with their merits and need-to-address challenges discussed. Then the unique properties (what) and potential applications of CMT in different learning algorithms/scenarios of spiking neural networks (why) are reviewed, including supervised rule, reinforcement one, dynamic vision with in-sensor computing, etc. Through exploiting the complementary structure-related novel functions, significant reduction of hardware consuming, enhancement of energy/efficiency ratio and other advantages have been gained, illustrating the alluring prospect of design technology co-optimization (DTCO) of CMT towards neuromorphic computing.

show abstract

“…Recently, a memristor analog crossbar circuit is used to emulate a single layer perceptron for the MNIST image classification problem (Kim et al, 2021 ). In Shi et al ( 2021 ), a circuit proposed to manage reward modulation is presented, setting the building blocks for implementation.…”

Section: Introductionmentioning

confidence: 99%

R-STDP Spiking Neural Network Architecture for Motion Control on a Changing Friction Joint Robotic Arm

et al. 2022

View full text Add to dashboard Cite

Neuromorphic computing is a recent class of brain-inspired high-performance computer platforms and algorithms involving biologically-inspired models adopting hardware implementation in integrated circuits. The neuromorphic computing applications have provoked the rise of highly connected neurons and synapses in analog circuit systems that can be used to solve today's challenging machine learning problems. In conjunction with biologically plausible learning rules, such as the Hebbian learning and memristive devices, biologically-inspired spiking neural networks are considered the next-generation neuromorphic hardware construction blocks that will enable the deployment of new analog in situ learning capable and energetic efficient brain-like devices. These features are envisioned for modern mobile robotic implementations, currently challenging to overcome the pervasive von Neumann computer architecture. This study proposes a new neural architecture using the spike-time-dependent plasticity learning method and step-forward encoding algorithm for a self tuning neural control of motion in a joint robotic arm subjected to dynamic modifications. Simulations were conducted to demonstrate the proposed neural architecture's feasibility as the network successfully compensates for changing dynamics at each simulation run.

show abstract

Exploiting Memristors for Neuromorphic Reinforcement Learning

Cited by 8 publications

References 20 publications

Building an Analog Circuit Synapse for Deep Learning Neuromorphic Processing

Building an Analog Circuit Synapse for Deep Learning Neuromorphic Processing

Complementary memtransistors for neuromorphic computing: How, what and why

R-STDP Spiking Neural Network Architecture for Motion Control on a Changing Friction Joint Robotic Arm

Contact Info

Product

Resources

About