S2N2: A FPGA Accelerator for Streaming Spiking Neural Networks

Khodamoradi, Alireza; Denolf, Kristof; Kastner, Ryan

doi:10.1145/3431920.3439283

Cited by 41 publications

(20 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The PEs are highly pipelined to achieve a high clock frequency and to improve parallelism. In contrast HLS-based approaches like the one of Fang et al [8] or S2N2 [39], our approach is agnostic to the CSNN's architecture and can thus be implemented on an ASIC as well. As the hardware utilization of a single convolution unit is so small, mutliple convolution units can be implemented in parallel, allowing easy scaling of throughput.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks

Sommer¹,

Özkan²,

Keszöcze³

et al. 2022

Preprint

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) compute in an event-based matter to achieve a more efficient computation than standard Neural Networks. In SNNs, neuronal outputs (i.e. activations) are not encoded with real-valued activations but with sequences of binary spikes. The motivation of using SNNs over conventional neural networks is rooted in the special computational aspects of spike based processing, especially the very high degree of sparsity of neural output activations. Well established architectures for conventional Convolutional Neural Networks (CNNs) feature large spatial arrays of Processing Elements (PEs) that remain highly underutilized in the face of activation sparsity. We propose a novel architecture that is optimized for the processing of Convolutional SNNs (CSNNs) that feature a high degree of activation sparsity. In our architecture, the main strategy is to use less but highly utilized PEs. The PE array used to perform the convolution is only as large as the kernel size, allowing all PEs to be active as long as there are spikes to process. This constant flow of spikes is ensured by compressing the feature maps (i.e. the activations) into queues that can then be processed spike by spike. This compression is performed in runtime using dedicated circuitry, leading to a self-timed scheduling. This allows the processing time to scale directly with the number of spikes. A novel memory organization scheme called memory interlacing is used to efficiently store and retrieve the membrane potentials of the individual neurons using multiple small parallel on-chip RAMs. Each RAM is hardwired to its PE, reducing switching circuitry and allowing RAMs to be located in close proximity to the respective PE. We implemented the proposed architecture on an FPGA and achieved a significant speedup compared to other implementations while needing less hardware resources and maintaining a lower energy consumption.

show abstract

Section: Discussionmentioning

confidence: 99%

“…In the future we plan to implement larger SNNs and also compare our results to non-spiking implementations. ASIC 0.001 98.0 SIES [18] FPGA 99.2 S2N2 [39] FPGA 98.5…”

Section: Discussionmentioning

confidence: 99%

Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks

Sommer¹,

Özkan²,

Keszöcze³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, their hardware design occupies many resources, making it difficult to be deployed on devices with small footprints. S2N2 [34], on the other hand, is a SIMD architecture with a high resource efficiency. The architecture was also evaluated on a two-dimensional image dataset.…”

Section: Related Workmentioning

confidence: 99%

E3NE: An End-to-End Framework for Accelerating Spiking Neural Networks with Emerging Neural Encoding on FPGAs

Gerlinghoff,

Wang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Compiler frameworks are crucial for the widespread use of FPGA-based deep learning accelerators. They allow researchers and developers, who are not familiar with hardware engineering, to harness the performance attained by domain-specific logic. There exists a variety of frameworks for conventional artificial neural networks. However, not much research effort has been put into the creation of frameworks optimized for spiking neural networks (SNNs). This new generation of neural networks becomes increasingly interesting for the deployment of AI on edge devices, which have tight power and resource constraints. Our end-to-end framework E3NE automates the generation of efficient SNN inference logic for FPGAs. Based on a PyTorch model and user parameters, it applies various optimizations and assesses trade-offs inherent to spike-based accelerators. Multiple levels of parallelism and the use of an emerging neural encoding scheme result in an efficiency superior to previous SNN hardware implementations. For a similar model, E3NE uses less than 50% of hardware resources and 20% less power, while reducing the latency by an order of magnitude. Furthermore, scalability and generality allowed the deployment of the large-scale SNN models AlexNet and VGG.

show abstract

“…Various accelerators have recently been proposed, which can be divided into two approaches based on their supported topologies [1]: general mesh and feedforward. For general mesh topology, large-scale neuromorphic hardware systems such as TrueNorth [2], Loihi [3], SpiNNaker [4], BrainScaleS [5], ODIN [6], µBrain [7] and DYNAPs [8] support a mesh of neurons with no particular topology by advanced routers and schedulers.…”

Section: Introductionmentioning

confidence: 99%

Sparse Compressed Spiking Neural Network Accelerator for Object Detection

Lien

Chang

2022

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

Spiking neural networks (SNNs), which are inspired by the human brain, have recently gained popularity due to their relatively simple and low-power hardware for transmitting binary spikes and highly sparse activation maps. However, because SNNs contain extra time dimension information, the SNN accelerator will require more buffers and take longer to infer, especially for the more difficult high-resolution object detection task. As a result, this paper proposes a sparse compressed spiking neural network accelerator that takes advantage of the high sparsity of activation maps and weights by utilizing the proposed gated oneto-all product for low power and highly parallel model execution. The experimental result of the neural network shows 71.5% mAP with mixed (1,3) time steps on the IVS 3cls dataset. The accelerator with the TSMC 28nm CMOS process can achieve 1024×576@29 frames per second processing when running at 500MHz with 35.88TOPS/W energy efficiency and 1.05mJ energy consumption per frame.

show abstract

S2N2: A FPGA Accelerator for Streaming Spiking Neural Networks

Cited by 41 publications

References 35 publications

Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks

Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks

E3NE: An End-to-End Framework for Accelerating Spiking Neural Networks with Emerging Neural Encoding on FPGAs

Sparse Compressed Spiking Neural Network Accelerator for Object Detection

Contact Info

Product

Resources

About