MB-CNN: Memristive Binary Convolutional Neural Networks for Embedded Mobile Devices

Chowdhury, Arjun Pal; Kulkarni, Pranav; Bojnordi, Mahdi Nazm

doi:10.3390/jlpea8040038

Cited by 17 publications

(14 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, more complex tasks like face recognition have been demonstrated (Yao et al, 2020). These similar network performances are often achieved at higher-energy efficiencies and make memristive device-based ANNs most useful for low-energy applications at the edge and in the IoT sector (Chowdhury et al, 2018) (Krestinskaya et al, 2020). A large variety of different types of memristive devices have been proposed for neuronal networks so far in the literature mimicking behavior of biological synapses like, e.g., long-term potentiation and depression (LTP/LTD) and even more complex aspects of synaptic plasticity like simple forms of spike timing dependent plasticity (STDP), but no optimal memristive device type has been identified yet.…”

Section: Introductionmentioning

confidence: 96%

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

et al. 2021

View full text Add to dashboard Cite

Memristive devices are novel electronic devices, which resistance can be tuned by an external voltage in a non-volatile way. Due to their analog resistive switching behavior, they are considered to emulate the behavior of synapses in neuronal networks. In this work, we investigate memristive devices based on the field-driven redox process between the p-conducting Pr0.7Ca0.3MnO3 (PCMO) and different tunnel barriers, namely, Al2O3, Ta2O5, and WO3. In contrast to the more common filamentary-type switching devices, the resistance range of these area-dependent switching devices can be adapted to the requirements of the surrounding circuit. We investigate the impact of the tunnel barrier layer on the switching performance including area scaling of the current and variability. Best performance with respect to the resistance window and the variability is observed for PCMO with a native Al2O3 tunnel oxide. For all different layer stacks, we demonstrate a spike timing dependent plasticity like behavior of the investigated PCMO cells. Furthermore, we can also tune the resistance in an analog fashion by repeated switching the device with voltage pulses of the same amplitude and polarity. Both measurements resemble the plasticity of biological synapses. We investigate in detail the impact of different pulse heights and pulse lengths on the shape of the stepwise SET and RESET curves. We use these measurements as input for the simulation of training and inference in a multilayer perceptron for pattern recognition, to show the use of PCMO-based ReRAM devices as weights in artificial neural networks which are trained by gradient descent methods. Based on this, we identify certain trends for the impact of the applied voltages and pulse length on the resulting shape of the measured curves and on the learning rate and accuracy of the multilayer perceptron.

show abstract

Section: Introductionmentioning

confidence: 96%

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A similar equation can be calculated for the XNOR operation which was done by Chowdhury et al in their system analysis of an equivalent concept. [ 23 ] In our approach, the voltage divider effect is used to retrieve the HD from the corresponding voltage drop of the shared electrode, and from that the bMAC result can easily be computed if needed.…”

Section: Concept and Hardware Realizationmentioning

confidence: 99%

In‐Memory Binary Vector–Matrix Multiplication Based on Complementary Resistive Switches

Ziegler

Waser

Wouters

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

With artificial neural networks (ANNs) becoming more and more powerful and with the slowdown of complementary metal-oxide-semiconductor (CMOS) scaling, the Von Neumann memory wall is becoming an increasingly prominent problem for ANN hardware systems. [1,2] Large neural networks especially suffer from this because not all computational information necessary can be stored in the cache memory, and costly communication with higher level storage is necessary. [3] In software, techniques such as pruning, weight reuse, or reducing the quantization are used to create less complex but still accurate ANNs. [4] A promising simplification from the hardware perspective is the aforementioned reduction in quantization due to its reduced computational complexity. The number of quantization levels can even go down to the bare minimum of two levels and results in binary neural networks (bNNs) which have been heavily studied in recent years. [5-8] These networks use the values 1 and À1 to encode their weights and activations during the inference step making them the most efficient ANNs possible to compute in hardware. [9] With ongoing improvements in prediction accuracy and the development of new hardware accelerators, these networks are auspicious candidates for computing ANNs on edge devices. Apart from the use of CMOS accelerators, new emerging technologies based on resistive switching devices can play a crucial role in this advancement. [10] These devices can mimic the synaptic weights in ANNs and, configured in a crossbar architecture, enable fast analog computations of vector-matrix multiplications, which are the main operations in ANNs. [11,12] One promising class of resistive switching devices relies on redox reactions and is therefore called redox-based resistive switching devices, also known as redox-based random access memory (ReRAM). [13] They are typically based on metaloxide-metal stacks and can change the conduction through the oxide layer based on electric signals applied to the metal electrodes. These changes in resistance from a high resistive state (HRS) to a low resistance state (LRS) and vice versa originate from ionic movements in the oxide layer and concurrent redox reactions. [14] There are already many realizations of ReRAM crossbar arrays used as accelerators for vector-matrix multiplication. [15-17] One common type uses Kirchoff 's current law to do the computation where the calculation result is encoded in the resulting current. To compute this result, the current has to be sensed, which becomes more difficult the lower the current is. This leads to a trade-off in the circuit design as a lower current reduces the energy consumption. Another commonly used architecture builds up a resistive voltage divider between a sense resistance and the resistive crossbar array. In this architecture, the result of the computation is encoded in the voltage drop of the voltage divider. [18] One design challenge of this architecture is the

show abstract

“…The main reason to apply DNN is the benefits obtained in many related areas, where the enhancing of signals represents important challenges. For example, to provide better speech recognition in adverse conditions or environments [14,15], and the implementation in low power consumption systems [16].…”

Section: Introductionmentioning

confidence: 99%

Improving Post-Filtering of Artificial Speech Using Pre-Trained LSTM Neural Networks

Coto-Jiménez

2019

Biomimetics

View full text Add to dashboard Cite

Several researchers have contemplated deep learning-based post-filters to increase the quality of statistical parametric speech synthesis, which perform a mapping of the synthetic speech to the natural speech, considering the different parameters separately and trying to reduce the gap between them. The Long Short-term Memory (LSTM) Neural Networks have been applied successfully in this purpose, but there are still many aspects to improve in the results and in the process itself. In this paper, we introduce a new pre-training approach for the LSTM, with the objective of enhancing the quality of the synthesized speech, particularly in the spectrum, in a more efficient manner. Our approach begins with an auto-associative training of one LSTM network, which is used as an initialization for the post-filters. We show the advantages of this initialization for the enhancing of the Mel-Frequency Cepstral parameters of synthetic speech. Results show that the initialization succeeds in achieving better results in enhancing the statistical parametric speech spectrum in most cases when compared to the common random initialization approach of the networks.

show abstract

MB-CNN: Memristive Binary Convolutional Neural Networks for Embedded Mobile Devices

Cited by 17 publications

References 73 publications

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

In‐Memory Binary Vector–Matrix Multiplication Based on Complementary Resistive Switches

Improving Post-Filtering of Artificial Speech Using Pre-Trained LSTM Neural Networks

Contact Info

Product

Resources

About