Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

Abstract

The massively parallel nature of biological information processing plays an important role due to its superiority in comparison to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.

Fast and energy-efficient neuromorphic deep learning with first-spike times

“…Due to variations in the manufacturing process, the realized circuits systematically deviate from each other (fixedpattern noise). Although these variations can be reduced by calibrating each circuit [72], considerable differences remain (standard deviation on the order of 5 % on BrainScaleS-2) and pose a challenge for possible neuromorphic algorithms -along with other features of physical model systems such as spike time jitter or spike loss [33,34,63,73].…”

“…Many attempts have been made to reconcile spiking neural networks with their abstract counterparts in terms of functionality, e.g., featuring spike-based inference models [28][29][30][31][32][33][34][35][36] and deep models trained on target spike times by shallow learning rules [37,38] or using spike-compatible versions of the error backpropagation algorithm [39][40][41]. Especially for tasks operating on static information, a particularly elegant way of utilizing the temporal aspect of exact spike times is the time-to-first-spike (TTFS) coding scheme [42].…”

“…In further software simulations, we show that our model deals well with various levels of substrate-induced distortions such as fixed-pattern noise and limited parameter precision and control, thus providing a rigorous algorithmic backbone for a wide range of neuromorphic substrates and applications. Such robustness with respect to imperfections of the underlying neuronal substrate represents an indispensable property for any network model aiming for biological plausibility and for every application geared towards physical computing systems [33,34,[60][61][62][63][64].…”

Brain-Inspired Hardware Solutions for Inference in Bayesian Networks

“…The BrainScaleS platform ( Schemmel et al, 2010 ), a physical-model neuromorphic device, emulates networks of spiking neurons. This platform is a mixed-signal neuromorphic system, using 180-nm CMOS technology for fabrication, on which Kungl et al (2019) proposed the first scalable implementation of sampling-based probabilistic inference with spiking networks. In order to sample from target distributions and hierarchical spiking networks with higher-dimensional input data, fully connected spiking networks have been trained.…”

“…Several complex floating-point calculations are required to estimate the probability of occurrence of a variable since the network is composed of various interacting causal variables ( Shim et al, 2017 ). Moreover, the high parallelism feature of Bayesian inference is not used efficiently in conventional computing systems (F. Kungl et al, 2019 ). Conventional systems need exact values throughout the computation, preventing the use of the stochastic computing paradigm that consumes less power ( Khasanvis et al, 2015a ).…”