Synapse cell optimization and back-propagation algorithm implementation in a domain wall synapse based crossbar neural network for scalable on-chip learning

Abstract

On-chip learning in spin orbit torque driven domain wall synapse based crossbar fully connected neural network (FCNN) has been shown to be extremely efficient in terms of speed and energy, when compared to training on a conventional computing unit or even on a crossbar FCNN based on other non-volatile memory devices. However there are issues with respect to scalability of the on-chip learning scheme in the domain wall synapse based FCNN. Unless the scheme is scalable, it will not be competitive with respect to training a neural network on a conventional computing unit for real applications. In this paper, we have proposed a modification in the standard gradient descent algorithm, used for training such FCNN, by including appropriate thresholding units. This leads to optimization of the synapse cell at each intersection of the crossbars and makes the system scalable. In order for the system to approximate a wide range of functions for data classification, hidden layers must be present and the backpropagation algorithm (extension of gradient descent algorithm for multi-layered FCNN) for training must be implemented on hardware. We have carried this out in this paper by employing an extra crossbar. Through a combination of micromagnetic simulations and SPICE circuit simulations, we hence show highly improved accuracy for domain wall syanpse based FCNN with a hidden layer compared to that without a hidden layer for different machine learning datasets.

Ferrimagnetic Synapse Devices for Fast and Energy-Efficient On-Chip Learning on An Analog-Hardware Neural Network

“…In-plane current flowing through the underlying heavy-metal layer (Pt) results in the injection of spin current into the ferromagnetic CoFe layer [1]- [3]. Hence, due to the spin-orbit torque, the domain wall moves, as captured in our micromagnetic simulation of this device on "mumax3" [20]- [22].…”

“…We also assume an interfacial Dzyalonshinskii Moriya Interaction (DMI) strength of 1.2 mJ/m 2 due to which the domain wall acquires Neel-type chirality. These simulation parameters have also been used in the experimentally benchmarked micromagnetic study of the heavy-metal/ferromagnet-bilayer device we consider here [21], [22], [46]. Fig.…”

“…2) System-Level Study: We next calculate the speed and energy consumption for on-chip learning (training in hardware) on a crossbar array of ferrimagnetic synapses, which we have simulated at the device level using micromagnetics as mentioned above. After that, we calculate the speed and energy consumption for on-chip learning on a crossbar array of ferromagnetic synapses [12], [13], [20]- [22]. We show that when the time for training is the same for both the crossbar arrays, energy consumption is four times (4X) lower in the ferrimagnetbased crossbar when compared to the ferromagnet-based crossbar.…”

“…Recently, along with memory devices, ferromagnetic domain-wall motion has been proposed as the working principle for synaptic devices in analog-hardware implementation of neural networks as well [12]- [22]. Neural networks currently form the most popular way to carry out machine-learning tasks for large-scale data [23].…”

“…In a ferromagnetic synapse device, each weight value corresponds to a different position of the domain wall in the device. To modulate the weight value, the domain wall is moved using current pulses [12], [13], [20]- [22]. If the ferromagnet-based synapse devices are replaced by their ferrimagnetic counterparts, then intuitively, faster domain-wall motion (in the ferrimagnetic system, compared to the ferromagnetic system) will enable the use of faster and lowerenergy-consuming current pulses for weight modulation and hence faster and more energy-efficient analog-crossbar-arraybased neural-network implementation.…”