Power optimizations in MTJ-based Neural Networks through Stochastic Computing

Abstract

Abstract-Artificial Neural Networks (ANNs) have found widespread applications in tasks such as pattern recognition and image classification. However, hardware implementations of ANNs using conventional binary arithmetic units are computationally expensive, energy-intensive and have large area overheads. Stochastic Computing (SC) is an emerging paradigm which replaces these conventional units with simple logic circuits and is particularly suitable for fault-tolerant applications. Spintronic devices, such as Magnetic Tunnel Junctions (MTJs), are capable of replacing CMOS in memory and logic circuits. In this work, we propose an energy-efficient use of MTJs, which exhibit probabilistic switching behavior, as Stochastic Number Generators (SNGs), which forms the basis of our NN implementation in the SC domain. Further, error resilient target applications of NNs allow us to introduce Approximate Computing, a framework wherein accuracy of computations is traded-off for substantial reductions in power consumption. We propose approximating the synaptic weights in our MTJ-based NN implementation, in ways brought about by properties of our MTJ-SNG, to achieve energy-efficiency. We design an algorithm that can perform such approximations within a given error tolerance in a single-layer NN in an optimal way owing to the convexity of the problem formulation. We then use this algorithm and develop a heuristic approach for approximating multi-layer NNs. To give a perspective of the effectiveness of our approach, a 43% reduction in power consumption was obtained with less than 1% accuracy loss on a standard classification problem, with 26% being brought about by the proposed algorithm.

SPINBIS: Spintronics-Based Bayesian Inference System With Stochastic Computing

“…In [35], an MTJ based analog-tostochastic converter is proposed for stochastic computation in vision chips. In [17], MTJ based stochastic computing is integrated into artificial neural network applications. However, the energy efficiency of their SBGs is relative low.…”

“…However, current analytical models of switching STT-MTJs [13]- [15] face the problem that the available integrated models do not link smoothly all regimes of current. An intermediate regime of current exists, where the physics of the switching is complex [16], [17] and not properly described by classical analytical models. Sun's model, the equation conventionally used to describe the high-current regime, actually diverges in this intermediate current regime.…”

Low Cost Hybrid Spin-CMOS Compressor for Stochastic Neural Networks

“…The basic unit is a neuron formulated as out = σ (X * W + bias) where σ is the activation function; X is the input of the neuron; W and bias are constant values in the classification which are obtained from training process. The MTJ based SNG [26] is used to generate bit-streams in the neuron architecture of Figure 8. The inputs are ANDed with the weighted bit-streams to obtain the intermediate products of matrix multiplications.…”

“…The spintronic components are quantitatively simulated using the device and material parameters summarized in Table 1. To provide a fair comparison, we use the MTJ-based SNGs of [26] to generate the required random numbers in all stochastic implementations. Four implementations, (#1) the conventional fixed-point binary implementation, (#2) the CMOS stochastic implementation with the bipolar format [16] [6], (#3) the CMOS implementation with the unipolar format [7] [8], and (#4) the hybrid spin-CMOS implementation are compared in terms of power, area and energy consumption.…”

Low-Cost Stochastic Hybrid Multiplier for Quantized Neural Networks

“…In these SNGs, certain patterns are generated sequentially to increase the accuracy of bit streams with short stream lengths. Moreover, by using spintronic devices, low-cost structures have been introduced to reduce the hardware cost of SNGs and to achieve high performance at a reasonable power and area cost [12,25]. Dickson et al [4] and Qian and Riedel [29] introduced two methods of adding stochastic bit streams.…”