Adaptive Precision CNN Accelerator Using Radix-X Parallel Connected Memristor Crossbars

Abstract

Neural processor development is reducing our reliance on remote server access to process deep learning operations in an increasingly edge-driven world. By employing inmemory processing, parallelization techniques, and algorithmhardware co-design, memristor crossbar arrays are known to efficiently compute large scale matrix-vector multiplications. However, state-of-the-art implementations of negative weights require duplicative column wires, and high precision weights using single-bit memristors further distributes computations. These constraints dramatically increase chip area and resistive losses, which lead to increased power consumption and reduced accuracy. In this paper, we develop an adaptive precision method by varying the number of memristors at each crosspoint. We also present a weight mapping algorithm designed for implementation on our crossbar array. This novel algorithm-hardware solution is described as the radix-X Convolutional Neural Network Crossbar Array, and demonstrate how to efficiently represent negative weights using a single column line, rather than double the number of additional columns. Using both simulation and experimental results, we verify that our radix-5 CNN array achieves a validation accuracy of 90.5% on the CIFAR-10 dataset, a 4.5% improvement over binarized neural networks whilst simultaneously reducing crossbar area by 46% over conventional arrays by removing the need for duplicate columns to represent signed weights.Index Terms-adaptive precision, algorithm hardware codesign, convolutional neural network (CNN), deep learning accelerator, low precision weight, memristor crossbar. I. INTRODUCTIONM ACHINE learning algorithms have become ubiquitous in the modern world, and are crucial in enabling computer systems which automatically update and improve with experience. This has opened up new frontiers in data analysis techniques. Deep learning refers to the use of a multilayered neural network where the sequence of layers between the input and output perform feature identification at various hierarchies, as inspired by an approximation of the neuronal connections within the brain [1]- [4]. A popular deep learning algorithm for structured data is the convolutional neural

Exploring Computing Paradigms for Electric Vehicles: From Cloud to Edge Intelligence, Challenges and Future Directions

“…Feasible methods include component shutdown, input filtering, early exit, and results caching. This concept can be likened to block-wise dropout [135,136]. Additionally, random gossip communication can help reduce unnecessary calculations and model updates.…”

Low Power In-Memory Implementation of Ternary Neural Networks with Resistive RAM-Based Synapse

“…Remarkably, this type of neural network can approach near-state-of-the-art performance on vision tasks [5]. One particularly investigated lead is to fabricate hardware BNNs with emerging memories such as resistive RAM or memristors [6]- [13]. The low memory requirements of BNNs, as well as their reliance on simple arithmetic operations, make them indeed particularly adapted for "in-memory" or "near-memory" computing approaches, which achieve superior energy-efficiency by avoiding the von-Neumann bottleneck entirely.…”

“…In particular, multiple groups investigate the implementation of BNN inference with resistive memory tightly integrated at the core of CMOS [6]- [13]. Usually, resistive memory stores the synaptic weights W ji .…”

Implementation of Ternary Weights with Resistive RAM Using a Single Sense Operation per Synapse

“…Remarkably, this type of neural network can achieve high accuracy on vision tasks [5]. One particularly investigated lead is to fabricate hardware BNNs with emerging memories such as resistive RAM or memristors [6]- [13]. The low memory requirements of BNNs, as well as their reliance on simple arithmetic operations, make them indeed particularly adapted for "in-memory" or "near-memory" computing approaches, which achieve superior energy-efficiency by avoiding the von Neumann bottleneck entirely.…”

“…Due to their reduced memory requirements, and reliance on simple arithmetic operations, BNNs are especially appropriate for in-or near-memory implementations. In particular, multiple groups investigate the implementation of BNN inference with resistive memory tightly integrated at the core of CMOS [6]- [13].…”