Analogue pattern recognition with stochastic switching binary CMOS-integrated memristive devices

Abstract

Biological neural networks outperform current computer technology in terms of power consumption and computing speed while performing associative tasks, such as pattern recognition. The analogue and massive parallel in-memory computing in biology differs strongly from conventional transistor electronics that rely on the von Neumann architecture. Therefore, novel bio-inspired computing architectures have been attracting a lot of attention in the field of neuromorphic computing. Here, memristive devices, which serve as non-volatile resistive memory, are employed to emulate the plastic behaviour of biological synapses. In particular, CMOS integrated resistive random access memory (RRAM) devices are promising candidates to extend conventional CMOS technology to neuromorphic systems. However, dealing with the inherent stochasticity of resistive switching can be challenging for network performance. In this work, the probabilistic switching is exploited to emulate stochastic plasticity with fully CMOS integrated binary RRAM devices. Two different RRAM technologies with different device variabilities are investigated in detail, and their potential applications in stochastic artificial neural networks (StochANNs) capable of solving MNIST pattern recognition tasks is examined. A mixed-signal implementation with hardware synapses and software neurons combined with numerical simulations shows that the proposed concept of stochastic computing is able to process analogue data with binary memory cells.

Stochastic binary synapses having sigmoidal cumulative distribution functions for unsupervised learning with spike timing-dependent plasticity

“…Stochastic operation of binary synapses is one possible solution 28 . Suri, et al showed that the probabilistic SET and RESET of binary memristors can be used for STDP-based learning 19 .…”

Self‐Selective Organic Memristor by Engineered Conductive Nanofilament Diffusion for Realization of Practical Neuromorphic System

“…As shown in Figure S10, Supporting Information, crossbar arrays of the organic memristor were used for a synapse layer. Based on the result in Figure 5e, we assumed the organic memristor was operated as a binary memory cell [43] with a broad read window for the perfect separation of resistance states in sensing operation, and three cells were formed for an artificial synapse with 3 bits of memory. In addition, the ideal vector-matrix multiplication operations which are importantly used in inference were assumed in the organic memristor crossbar array.…”

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

“…From the literature, two main pathways are identified, these are the single-device and the compound-device architectures. On the side of single device architectures, extensive studies exist on the variability phenomenon both for d2d and c2c aspects ( Jo et al, 2009 ; Suri et al, 2013 ; Yu et al, 2013a ; Naous et al, 2016 ; Wenger et al, 2019 ; Zahari et al, 2020 ). The idea to incorporate multiple resistive devices into a synapse has been introduced earlier ( Gaba et al, 2013 ; Bill and Legenstein, 2014 ; Hu et al, 2014 ; Singha et al, 2014 ; Boybat et al, 2018 , 2019 ; Payvand et al, 2018 , 2019 ).…”

“…Common for previous works published on the subject of stochastic switching of ReRAM devices in neural networks is the utilization of behavioral models. These lead to voltage-dependent switching probability models such as the Poisson distribution ( Jo et al, 2009 ; Gaba et al, 2013 ; Naous et al, 2016 ; Payvand et al, 2018 ; Zahari et al, 2020 ), sigmoidal distribution ( Wenger et al, 2019 ), Gaussian distribution ( Yu et al, 2013b ) and lognormal distribution ( Medeiros-Ribeiro et al, 2011 ; Hu et al, 2014 ), and even linear dependence ( Singha et al, 2014 ). By definition, these models only capture the minimal required behavioral aspects and possess little to no predictive character for any setup modification.…”