Bayesian neural networks using magnetic tunnel junction-based probabilistic in-memory computing

Abstract

Bayesian neural networks (BNNs) combine the generalizability of deep neural networks (DNNs) with a rigorous quantification of predictive uncertainty, which mitigates overfitting and makes them valuable for high-reliability or safety-critical applications. However, the probabilistic nature of BNNs makes them more computationally intensive on digital hardware and so far, less directly amenable to acceleration by analog in-memory computing as compared to DNNs. This work exploits a novel spintronic bit cell that efficiently and compactly implements Gaussian-distributed BNN values. Specifically, the bit cell combines a tunable stochastic magnetic tunnel junction (MTJ) encoding the trained standard deviation and a multi-bit domain-wall MTJ device independently encoding the trained mean. The two devices can be integrated within the same array, enabling highly efficient, fully analog, probabilistic matrix-vector multiplications. We use micromagnetics simulations as the basis of a system-level model of the spintronic BNN accelerator, demonstrating that our design yields accurate, well-calibrated uncertainty estimates for both classification and regression problems and matches software BNN performance. This result paves the way to spintronic in-memory computing systems implementing trusted neural networks at a modest energy budget.

Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks

“…Two recent publications have proposed new devices where the inherent probability distribution of their resistance can be tuned. These solutions, which exploit two-dimensional materials 24 and magnetic devices 25 , were validated with simulations of Bayesian neural networks.…”

“…• Each synapse is implemented using two memristors that are programmed independently, allowing the partial decorrelation of mean values and standard deviations of synaptic weights. This idea was presented in simulation studies 24,25 .…”

Probabilistic Photonic Computing with Chaotic Light

“…For electronic crossbar arrays, memristors serve as the main in-memory computation element due their tunable conductance. Simultaneously, programming and reading the conductance of a memristor is a stochastic process due to inherent randomness of the switching process in addition to drifts and instabilities 20,21 . Since the randomness is programmable by deploying multiple memristors for a single matrix weight, it can be deployed for Bayesian inference.…”

Graphene-Based Artificial Dendrites for Bio-Inspired Learning in Spiking Neuromorphic Systems

“…The requirements for ANN and SNN devices depend on their intended use, e.g., CMOS-compatible devices for state-of-the-art electronics and biocompatible devices for use in health. Many devices have been proposed for building these systems; − however, only a subset meets the performance requirements, − few are biocompatible, − and fewer leverage advanced biological behavior such as that of dendrites. − In biological systems, dendrites branch out from the neuronal body and process incoming spikes into nonspiking spatiotemporal signals. For example, the leaky integrate-and-fire model of neuronal behavior is composed of one leaky recurrent dendrite and a soma for an activation threshold.…”