Binary Weighted Memristive Analog Deep Neural Network for Near-Sensor Edge Processing

Krestinskaya, Olga; James, Alex Pappachen

doi:10.1109/nano.2018.8626224

Cited by 28 publications

(17 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The summary of these architectures is shown in Table I. Also, there are several other memristive architectures proposed in the recent years, which are less common and not considered in this paper, such as Probabilistic Neural Networks [88], [89] and Binarized Neural Networks [90]. 1) One layer neural network with learning: The structure of one-layer ANN with learning is similar to the feed-forward neural network but contains the learning phase.…”

Section: Neuromorphic Architectures a Neural Network Architecturesmentioning

confidence: 99%

Neuromemristive Circuits for Edge Computing: A Review

Krestinskaya

James

Chua

2020

IEEE Trans. Neural Netw. Learning Syst.

228

120

View full text Add to dashboard Cite

The volume, veracity, variability and velocity of data produced from the ever increasing network of sensors connected to Internet pose challenges for power management, scalability and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and show the advantages, drawbacks and open problems in the field of neuro-memristive circuits for edge computing.

show abstract

Section: Neuromorphic Architectures a Neural Network Architecturesmentioning

confidence: 99%

Neuromemristive Circuits for Edge Computing: A Review

Krestinskaya

James

Chua

2020

IEEE Trans. Neural Netw. Learning Syst.

228

120

View full text Add to dashboard Cite

show abstract

“…A neural network which uses any combination of binary weight or hard threshold activation functions is typically known as BNN. There have been several successful implementations of BNN algorithms in software [26], [27], [28], [29] and an attempt to implement it on hardware [30], [31], [32]. The analog hardware implementation of BNN system with learning remains as an open problem [33], [32].…”

Section: Background a Learning Algorithms And Biologically Inspimentioning

confidence: 99%

“…There have been several successful implementations of BNN algorithms in software [26], [27], [28], [29] and an attempt to implement it on hardware [30], [31], [32]. The analog hardware implementation of BNN system with learning remains as an open problem [33], [32]. MNN is a systematic arrangement of the artificial neural networks that can process information from different data sources to perform data fusion and classification.…”

Section: Background a Learning Algorithms And Biologically Inspimentioning

confidence: 99%

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Krestinskaya

Salama

James

2019

IEEE Trans. Circuits Syst. I

128

View full text Add to dashboard Cite

The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is done using TSMC 180nm CMOS process models, and TiO2 based memristor models. The application level validations of the system are done using XOR problem, MNIST character and Yale face image databases.

show abstract

“…Sometimes it is used in a 1 transistor 1 RRAM (1T1R) configuration, to avoid unwanted or sneak current paths. In [15], authors have presented a memristor-based implementation of a BNN able to achieve both high accuracy on MNIST and IRIS dataset and low power consumption. In some others, improvements in memristor architectures have been proposed that enable multiple bits per cell.…”

Section: A Quick Overviewmentioning

confidence: 99%

“…Some of them are considering the binary approximations, choosing an implementation based on emerging technologies. Some works [12,13,26,27] are based on MTJ technology while [15][16][17][18]28,29] have used RRAM. In each of these works the resistive element is used to perform simple logical operations based on current sensing technique.…”

Section: Nn Implementations Based On Lim Conceptmentioning

confidence: 99%

Logic-in-Memory Computation: Is It Worth It? A Binary Neural Network Case Study

Coluccio

Vacca

Turvani

2020

JLPEA

View full text Add to dashboard Cite

Recently, the Logic-in-Memory (LiM) concept has been widely studied in the literature. This paradigm represents one of the most efficient ways to solve the limitations of a Von Neumann’s architecture: by placing simple logic circuits inside or near a memory element, it is possible to obtain a local computation without the need to fetch data from the main memory. Although this concept introduces a lot of advantages from a theoretical point of view, its implementation could introduce an increasing complexity overhead of the memory itself, leading to a more sophisticated design flow. As a case study, Binary Neural Networks (BNNs) have been chosen. BNNs binarize both weights and inputs, transforming multiply-and-accumulate into a simpler bitwise logical operation while maintaining high accuracy, making them well-suited for a LiM implementation. In this paper, we present two circuits implementing a BNN model in CMOS technology. The first one, called Out-Of-Memory (OOM) architecture, is implemented following a standard Von Neumann structure. The same architecture was redesigned to adapt the critical part of the algorithm for a modified memory, which is also capable of executing logic calculations. By comparing both OOM and LiM architectures we aim to evaluate if Logic-in-Memory paradigm is worth it. The results highlight that LiM architectures have a clear advantage over Von Neumann architectures, allowing a reduction in energy consumption while increasing the overall speed of the circuit.

show abstract

Binary Weighted Memristive Analog Deep Neural Network for Near-Sensor Edge Processing

Cited by 28 publications

References 12 publications

Neuromemristive Circuits for Edge Computing: A Review

Neuromemristive Circuits for Edge Computing: A Review

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Logic-in-Memory Computation: Is It Worth It? A Binary Neural Network Case Study

Contact Info

Product

Resources

About