On-chip learning of a domain-wall-synapse-crossbar-array-based convolutional neural network

Abstract

Domain-wall-synapse-based crossbar arrays have been shown to be very efficient, in terms of speed and energy consumption, while implementing fully connected neural network (FCNN) algorithms for simple data-classification tasks, both in inference and on-chip-learning modes. But for more complex and realistic data-classification tasks, convolutional neural networks (CNN) need to be trained through such crossbar arrays. In this paper, we carry out device-circuit-system co-design and co-simulation of on-chip learning of a CNN using a domain-wall-synapse-based crossbar array. For this purpose, we use a combination of micromagnetic-physics-based synapse-device modeling, SPICE simulation of a crossbar-array circuit using such synapse devices, and system-level-coding using a high-level language. In our design, each synaptic weight of the convolutional kernel is considered to be of 15 bits; one domain-wall-synapse crossbar array is dedicated to the 5 least significant bits (LSBs), and two crossbar arrays are dedicated to the other bits. The crossbar arrays accelerate the matrix vector multiplication (MVM) operation involved in the forward computation of the CNN. The synaptic weights of the LSB crossbar are updated after forward computation on every training sample, while the weights of the other crossbars are updated after forward computation on 10 samples, to achieve on-chip learning. We report high classification-accuracy numbers for different machine-learning data sets using our method. We also carry out a study of how the classification accuracy of our designed CNN is affected by device-to-device variations, cycle-to-cycle variations, bit precision of the synaptic weights, and the frequency of weight updates.

Demonstration of Synaptic Behavior in a Heavy-Metal-Ferromagnetic-Metal-Oxide-Heterostructure-Based Spintronic Device for On-Chip Learning in Crossbar-Array-Based Neural Networks

“…Recently, in some other reports, it has been shown that synaptic noise during training can improve classification accuracy, , and it has also happened here in the case of Fashion-MNIST. It is to be noted that in all the cases presented here that involve two devices per synapse cell, the weight of the second device is updated by one bit when that of the first device reaches its maximum or minimum value …”

“…It is to be noted that in all the cases presented here that involve two devices per synapse cell, the weight of the second device is updated by one bit when that of the first device reaches its maximum or minimum value. 70 Estimating Energy and Speed for On-Chip Learning. We next estimate the energy and speed for on-chip learning of an FCNN on crossbar arrays based on our spintronic synapses.…”

Energy-efficient on-chip learning for a fully connected neural network using domain wall device

“…The TI to be used in realistic SOT-magnetic random-access memory (MRAM) must have a high spin hall angle and high conductivity thus, BiSb solves both conditions [19]. The free layer contains the weights corresponding to the conductance of DW which can be tuned by either a write current pulse or a magnetic field [4]. The synaptic characteristics of DW are linear and symmetric which makes the designing of the peripheral circuit very simple [12].…”

“…Among spintronic devices, magnetic domain wall (DW) structures require low driving current and offer high storage capacity [2]. These also find applications in implementing energy-efficient artificial neural networks (ANN) because of their superior performance in terms of speed and energy efficiency crossbar solution for solving image classification, speech recognition, machine translation, and other problems faster [3,4]. Moreover, ANNs are computational units that have advanced significantly in recent years and are a subset of artificial intelligence (AI) to solve classification and prediction problems [5].…”

Quantized Magnetic Domain Wall Synapse for Efficient Deep Neural Networks

“…Several methods have been adopted in the literature to address these issues. Multiple devices per synapse have been used to increase the precision and address non-linearity of the conductance response [41][42][43]. Bit-slicing technique [44] is used to slice the input and weight matrices into several smaller bit slices.…”