SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions

Mauro, Alfio Di; Prasad, Arpan Suravi; Huang, Zhikai; Spallanzani, Matteo; Conti, Francesco; Benini, Luca

doi:10.23919/date54114.2022.9774552

Cited by 14 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accelerator domain is optimized for ML-based perception. It comprises two compute engines: i) an all-digital sparse neural engine (SNE) [16], tailored for sparse neuromorphic workloads, specifically SNNs. SNE is designed for streamlined interaction with event-based sensors generating spikes, i.e., binary and time-stamped input events, that SNE processes in an event-driven fashion, and ii) CUTIE, designed for maximum inference energy efficiency and low latency by exploiting extreme quantization in TNNs to perform convolutions with a completely unrolled data path.…”

Section: Cutie (Extremementioning

confidence: 99%

Kraken: An Open-Source RISC-V SoC for Ultra-Low Power Multi-Modal Perception

Potocnik,

Mauro,

Leitner

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce Kraken, a highly flexible heterogeneous system-on-chip (SoC) fabricated in 22 FDX technology demonstrating leading-edge energy efficiency and computational capabilities for ultra-low power extreme-edge applications. Inspired by the multi-modal perception system of mammalians, Kraken supports multi-modal sensing and perception via event-based and sampling-based sensor interfaces coupled with a spiking neural network accelerator (SNE) and a ternary neural network accelerator (CUTIE), respectively. An octa-core 32-bit RISC-V cluster with ML and DSP extensions enables sensor fusion and flexible parallel data processing. A fabric controller (FC) featuring a dedicated core manages all compute subsystems and low computation intensity tasks. We analyzed Kraken’s performance and efficiency: SNE consumes 2.6 pJ/SOp and 470 μJ/inference for eye gaze prediction with 64% accuracy. CUTIE achieves 590 TOp/s/W and 2.72 μJ/inference for foveated object classification with 88% accuracy. The RISC-V cluster consumes 0.45 pJ per 8-bit MAC operation, and 12.01 μJ/inference for eye tracking with a mean average error of 0.24. All of Kraken’s subsystems improve upon the state-of-the-art (SOA), while the heterogenous architecture offers extreme flexibility in supporting ultra-low power multi-modal AI fusion applications.

show abstract

Section: Cutie (Extremementioning

confidence: 99%

Kraken: An Open-Source RISC-V SoC for Ultra-Low Power Multi-Modal Perception

Potocnik,

Mauro,

Leitner

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The third type of neuromorphic hardware follows the scheme of the ANN accelerator design except for constructing dedicated hardware for synaptic operations and explores optimal dataflow for SNNs specifically [20]- [26]. These types of work require less area cost and achieve higher computing resource utilization.…”

Section: Related Workmentioning

confidence: 99%

“…Compared with [21], FireFly achieves higher accuracy and a ×6 inference speed up on CIFAR10 dataset. Compared with ASIC design [26], FireFly achieves a ×2 speed up and similar accuracy on DVS-Gesture dataset. Note that our SNN models are considerably bigger and deeper than the listed benchmarks.…”

Section: Benchmark Evaluationsmentioning

confidence: 99%

FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks

Li¹,

Shen²,

Zhao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of spiking neural networks has become more complex, and the performance gap with artificial neural networks has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources by using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53TSOP/s at 300MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

show abstract

“…The approach is motivated by our long-term research interest in validating the accuracy and profiling the execution of event-based techniques for future implementation onto event-based computing platforms as an alternative to the dominant DL models relying on matrix multiplication on temporal data buffers [3], [5]. In contrast, event-based computing promises reduced computation latency and energy consumption [17]- [19].…”

Section: Introductionmentioning

confidence: 99%

Event-based Estimation of Hand Forces from High-Density Surface EMG on a Parallel Ultra-Low-Power Microcontroller

Zanghieri,

Rapa,

Orlandi

et al. 2024

IEEE Sensors J.

View full text Add to dashboard Cite

parallelism shortens latency Real-time embedded force estimation • accurate & all-fingers • robust cross-day High-Density sEMG Spike trains parallel ultra-low power MCU LIF Neurons 𝐝𝒙 𝐝𝒕 = 𝒙𝐝𝐫𝐢𝐯𝐞 − 𝒙 𝝉 deployment of selected LIFs' dynamics and fitted inference heuristic feature selection reduces computation workload fit of linear readout Abstract-Modeling hand kinematics and dynamics is a key goal for research on Human-Machine Interfaces, with surface electromyography (sEMG) being the most commonly used sensing modality. Though underresearched, sEMG regression-based modeling of hand movements and forces is promising for finer control than allowed by mapping to fixed gestures. We present an event-based sEMG encoding for multi-finger force estimation implemented on a microcontroller unit (MCU). We are the first to target the HYSER High-Density (HD)-sEMG dataset in multi-day conditions closest to a real scenario without a fixed force pattern. Our Mean Absolute Error of (8.4 ± 2.8)% of the Maximum Voluntary Contraction (MVC) is on par with State-of-the-Art (SoA) works on easier settings such as within-day, single-finger, or fixed-exercise. We deploy our solution for HYSER's hardest task on a parallel ultra-low power MCU, getting an energy consumption below 6.5 uJ per sample, 2.8× to 11× more energy-efficient than SoA single-core solutions, and a latency below 280 us per sample, shorter than HYSER's HD-sEMG sampling period, thus compatible with real-time operation on embedded devices.

show abstract

SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions

Cited by 14 publications

References 21 publications

Kraken: An Open-Source RISC-V SoC for Ultra-Low Power Multi-Modal Perception

Kraken: An Open-Source RISC-V SoC for Ultra-Low Power Multi-Modal Perception

FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks

Event-based Estimation of Hand Forces from High-Density Surface EMG on a Parallel Ultra-Low-Power Microcontroller

Contact Info

Product

Resources

About