Pattern representation and recognition with accelerated analog neuromorphic systems

Petrovici, Mihai A.; Schmitt, Sebastian; Klähn, Johann; Stöckel, David; Schroeder, Anna; Bellec, Guillaume; Bill, Johannes; Breitwieser, Oliver; Bytschok, Ilja; Grübl, Andreas; Güttler, Maurice; Hartel, Andreas; Hartmann, Stephan; Husmann, Dan; Husmann, Kai; Jeltsch, Sebastian; Karasenko, Vitali; Kleider, Mitja; Koke, Christoph; Kononov, Alexander; Mauch, Christian; Müller, Eric; Müller, Paul; Partzsch, Johannes; Pfeil, Thomas; Schiefer, Stefan; Scholze, Stefan; Subramoney, Anand; Thanasoulis, Vasilis; Vogginger, Bernhard; Legenstein, Robert; Maass, Wolfgang; Schüffny, René; Mayr, Christian; Schemmel, Johannes; Meier, K.

doi:10.1109/iscas.2017.8050530

Cited by 14 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In smaller networks, synaptic weight resolution is a critical performance modifier (Petrovici et al, 2017b). However, the penalty imposed by a limited synaptic weight resolution is known to decrease for larger deep networks with more and larger hidden layers, both spiking and non-spiking (Courbariaux et al, 2015; Petrovici et al, 2017a). Furthermore, the successor system (BrainScaleS-2, Aamir et al, 2016) is designed with a 6-bit weight resolution.…”

Section: Discussionmentioning

confidence: 99%

“…Previous small-scale studies of sampling on accelerated mixed-signal neuromorphic hardware include (Petrovici et al, 2015, 2017a,b). An implementation of sampling with spiking neurons and its application to the MNIST dataset was shown in Pedroni et al (2016) using the fully digital, real-time TrueNorth neuromorphic chip (Merolla et al, 2014).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

et al. 2019

View full text Add to dashboard Cite

The massively parallel nature of biological information processing plays an important role due to its superiority in comparison to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In contrast to the precise but resource hungry frame-based ANN approaches, our network is based on a spike code and therefore potentially very energy efficient if implemented on a neuromorphic hardware platform. Differences in timescale between inputs and hardware can be a problem for neuromorphic systems if they shall operate on natural stimuli in real time and the timescale of a neuromorphic system is often a design choice depending on the potential application (see for instance Qiao et al, 2015 ; Petrovici et al, 2017 for a real time and faster than real time analog neuromorphic hardware framework). Due to the fully event-based nature of our architecture, our network circumvents this problem and is able to operate on any time scales with computations being only driven by external input.…”

Section: Discussion and Outlookmentioning

confidence: 99%

Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP

Thiele

Bichler

Dupret

2018

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Learning of hierarchical features with spiking neurons has mostly been investigated in the database framework of standard deep learning systems. However, the properties of neuromorphic systems could be particularly interesting for learning from continuous sensor data in real-world settings. In this work, we introduce a deep spiking convolutional neural network of integrate-and-fire (IF) neurons which performs unsupervised online deep learning with spike-timing dependent plasticity (STDP) from a stream of asynchronous and continuous event-based data. In contrast to previous approaches to unsupervised deep learning with spikes, where layers were trained successively, we introduce a mechanism to train all layers of the network simultaneously. This allows approximate online inference already during the learning process and makes our architecture suitable for online learning and inference. We show that it is possible to train the network without providing implicit information about the database, such as the number of classes and the duration of stimuli presentation. By designing an STDP learning rule which depends only on relative spike timings, we make our network fully event-driven and able to operate without defining an absolute timescale of its dynamics. Our architecture requires only a small number of generic mechanisms and therefore enforces few constraints on a possible neuromorphic hardware implementation. These characteristics make our network one of the few neuromorphic architecture which could directly learn features and perform inference from an event-based vision sensor.

show abstract

“…Most of these use a spiking, pre-trained approach, i.e., the networks are trained in either ANN or SNN fashion, then in the case of ANNs, converted to SNNs, and implemented on the spiking neuromorphic hardware. Examples include TrueNorth (Esser et al, 2015 , 2016 ), SpiNNaker 1 (Jin et al, 2010 ), the BrainScaleS system (Petrovici et al, 2017 ; Schmitt et al, 2017 ), or the Zurich subthreshold systems (Indiveri et al, 2015 ). Of these, only the last one incorporates some learning, i.e., the last layer of the deep SNN is subject to online supervised learning, with the other layers having pretrained fixed weights.…”

Section: Discussionmentioning

confidence: 99%

Memory-Efficient Deep Learning on a SpiNNaker 2 Prototype

et al. 2018

View full text Add to dashboard Cite

The memory requirement of deep learning algorithms is considered incompatible with the memory restriction of energy-efficient hardware. A low memory footprint can be achieved by pruning obsolete connections or reducing the precision of connection strengths after the network has been trained. Yet, these techniques are not applicable to the case when neural networks have to be trained directly on hardware due to the hard memory constraints. Deep Rewiring (DEEP R) is a training algorithm which continuously rewires the network while preserving very sparse connectivity all along the training procedure. We apply DEEP R to a deep neural network implementation on a prototype chip of the 2nd generation SpiNNaker system. The local memory of a single core on this chip is limited to 64 KB and a deep network architecture is trained entirely within this constraint without the use of external memory. Throughout training, the proportion of active connections is limited to 1.3%. On the handwritten digits dataset MNIST, this extremely sparse network achieves 96.6% classification accuracy at convergence. Utilizing the multi-processor feature of the SpiNNaker system, we found very good scaling in terms of computation time, per-core memory consumption, and energy constraints. When compared to a X86 CPU implementation, neural network training on the SpiNNaker 2 prototype improves power and energy consumption by two orders of magnitude.

show abstract

Pattern representation and recognition with accelerated analog neuromorphic systems

Cited by 14 publications

References 8 publications

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP

Memory-Efficient Deep Learning on a SpiNNaker 2 Prototype

Contact Info

Product

Resources

About