Memristors Empower Spiking Neurons With Stochasticity

Abstract

Recent theoretical studies have shown that probabilistic spiking can be interpreted as learning and inference in cortical microcircuits. This interpretation creates new opportunities for building neuromorphic systems driven by probabilistic learning algorithms. However, such systems must have two crucial features: 1) the neurons should follow a specific behavioral model, and 2) stochastic spiking should be implemented efficiently for it to be scalable. This paper proposes a memristor-based stochastically spiking neuron that fulfills these requirements. First, the analytical model of the memristor is enhanced so it can capture the behavioral stochasticity consistent with experimentally observed phenomena. The switching behavior of the memristor model is demonstrated to be akin to the firing of the stochastic spike response neuron model, the primary building block for probabilistic algorithms in spiking neural networks. Furthermore, the paper proposes a neural soma circuit that utilizes the intrinsic nondeterminism of memristive switching for efficient spike generation. The simulations and analysis of the behavior of a single stochastic neuron and a winner-take-all network built of such neurons and trained on handwritten digits confirm that the circuit can be used for building probabilistic sampling and pattern adaptation machinery in spiking networks. The findings constitute an important step towards scalable and efficient probabilistic neuromorphic platforms.

Neuromemristive Circuits for Edge Computing: A Review

“…5 (i). Memristor is arranged in parallel with original simple neuron circuit consisting of membrane resistor R m and capacitor C m [84]. The variable threshold of the memristor allows to randomize the firing threshold of the neuron and ensures random neuron spiking behavior.…”

“…The stochasticity implies the probabilistic behavior of neurons or synapses and represents the biological concept of the importance of neural noise during the information processing in the brain. In [84], the stochasticity is introduced to the simple Spiking WTA architecture shown in Fig. 13, where the output is determined by the first firing neuron from the output neurons.…”

“…in [158]. This model is based on drift mechanism of ionized dopants and emulates T iO 2 memristor [84]. The linear relationship between the voltage and current in the memristor can be described as v(t) = (R ON × x(t) + R OF F (1x(t))) × i(t), where x(t) = w(t)/D and D is a width of the device and w(t) is a width of a doped region at a particular time [170].…”

Hierarchical Temporal Memory Using Memristor Networks: A Survey

“…In addition, the real memristive devices are not always accurate and involve the problem of switching stochasticity. Two real memristors of the same technology can react differently to the same voltage applied across the memristor for the switching [47]. The switching time and level may vary or, according to the theories of probabilistic behavior of the memristor, the switching may not occur.…”

Memristor-based neural networks: Synaptic versus neuronal stochasticity

“…The network performance showed high accuracy ranges and robustness to several characteristic metrics variation. 22 Spike-timing dependent plasticity (STDP) learning rule was applied in the simulations allowing the synapses to adapt in a more analog manner to the input images as shown in Figure 4a. Alternatively in the synaptic stochasticity, a stochastic learning rule was applied with binary synaptic weights that were also confined and abstracted into two levels only as depicted in the black and white images in Figure 4b.…”