Analytical Assessment of the Suitability of Multicast Communications for the SpiNNaker Neuromimetic System

Abstract

Abstract-SpiNNaker is a custom-made architecture designed to model large-scale spiking neural nets. One of the most significant characteristics of neural nets is their extreme communication needs; each neuron propagates its activation to thousands of other neurons. This paper shows analytical proof that the novel multicast router in SpiNNaker is a better solution for simulating neural nets than more powerful point-to-point routers such as those found on datacentres or high performance computing systems, even when it has significantly lower requirements in terms of complexity, area and power. First, we characterised the utilization of resources required by both multicast and unicast networks. Then we derived the bandwidth needs of different communication architectures. Finally, we derived the amount of neurons the different networks can support. From these analyses we determined that the multicast communications adopted in SpiNNaker will be able to support the target application under the expected operation conditions.

Randomly Spiking Dynamic Neural Fields

“…For instance the SpiNNaker chip [Jin et al 2010] is likely to be powerful in recurrent connectivity, given the good performance of their event-address mapping network (in Navaridas et al [2012], the chip is able to simulate a spiking neural networks of 2000 spiking neurons with all-to-all connections in the worst case scenario).…”

Understanding Computational Costs of Cellular-Level Brain Tissue Simulations Through Analytical Performance Models

“…This can be directly attributed to the fact that network latency introduces a performance overhead that increases linearly with the number of distributed ranks. Thus we conclude that large-scale simulations are dominated by the latency of the collective communication, and that investigating spike communication strategies such as neighbourhood collectives (Jordan et al 2018), nonblocking point-to-point schemes (Ananthanarayanan and Modha 2007), asynchronous execution (Magalhaes et al 2019b) or custom hardware (Navaridas et al 2012) will be essential to reach brain-scale simulations.…”

“…In distributed simulations we identified the network latency, and not the network bandwidth, as the major bottleneck for scaling to very large networks or very large cluster sizes. This provides a new motivation and justification for the extensive efforts described in Navaridas et al (2012) in designing a specific communication infrastructure for the SpiNNaker neuromorphic system.…”

“…An analysis showed that, in the context of real-time simulations, SpiNNaker is inherently limited in its scope to a restricted subset of synaptic formalism, because the single node's memory bandwidth puts a hard limit on the number of synaptic parameters that can be streamed at every timestep (Painkras et al 2013). In designing the network for SpiNNaker, a performance model was used to determine the ideal topology and network connectivity (Navaridas et al 2012), proving that performance modeling can be an extremely valuable tool in optimization and co-design for brain tissue simulations.…”

“…The level of diversity means that design tradeoffs will inevitably require very restrictive decisions about the scale, model and configuration of the target simulation, such as e.g. (Navaridas et al 2012;Knight and Nowotny 2018). For example, in the context of accelerating small networks of neurons up to real-time, we predict that I-based models would benefit from architectures with high memory level parallelism and a simple cache hierarchy, while for G-based models the cache bandwidth as well as the throughput of CPU arithmetic operations must be improved concurrently in order observe the desired speedup.…”