Neuromorphic Data Augmentation for Training Spiking Neural Networks

Li, Yuhang; Kim, Youngeun; Park, Hyoungseob; Geller, Tamar; Panda, Priyadarshini

doi:10.1007/978-3-031-20071-7_37

Cited by 42 publications

(15 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al [55] propose several randomized geometric augmentations for training SNNs. These include common techniques such as horizontal flip, translation, and rotation; as well as other unique techniques such as cutout, shear, and CutMix.…”

Section: Augmentation Methods For Event-based Visionmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

Event-based vision is a novel perception modality that offers several advantages, such as high dynamic range and robustness to motion blur. In order to process events in batches and utilize modern computer vision deep-learning architectures, an intermediate representation is required. Nevertheless, constructing an effective batch representation is non-trivial. In this paper, we propose a novel representation for event-based vision, called the compact spatio-temporal representation (CSTR). The CSTR encodes an event batch's spatial, temporal, and polarity information in a 3-channel image-like format. It achieves this by calculating the mean of the events' timestamps in combination with the event count at each spatial position in the frame. This representation shows robustness to motion-overlapping, high event density, and varying event-batch durations. Due to its compact 3-channel form, the CSTR is directly compatible with modern computer vision architectures, serving as an excellent choice for deploying event-based solutions. In addition, we complement the CSTR with an augmentation framework that introduces randomized training variations to the spatial, temporal, and polarity characteristics of event data. Experimentation over different object and action recognition datasets shows that the CSTR outperforms other representations of similar complexity under a consistent baseline. Further, the CSTR is made more robust and significantly benefits from the proposed augmentation framework, considerably addressing the sparseness in event-based datasets.

show abstract

Section: Augmentation Methods For Event-based Visionmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We use VGG16 (Simonyan and Zisserman, 2015 ) and ResNet19 (He et al, 2016 ). For both architectures, we use the scaled-up channel size following previous SNN works (Zheng et al, 2020 ; Li et al, 2022b ). We train the SNNs with 128 batch samples using SGD optimizer with momentum 0.9 and weight decay 5e-4.…”

Section: Methodsmentioning

confidence: 99%

Sharing leaky-integrate-and-fire neurons for memory-efficient spiking neural networks

Kim,

Li,

Moitra

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) have gained increasing attention as energy-efficient neural networks owing to their binary and asynchronous computation. However, their non-linear activation, that is Leaky-Integrate-and-Fire (LIF) neuron, requires additional memory to store a membrane voltage to capture the temporal dynamics of spikes. Although the required memory cost for LIF neurons significantly increases as the input dimension goes larger, a technique to reduce memory for LIF neurons has not been explored so far. To address this, we propose a simple and effective solution, EfficientLIF-Net, which shares the LIF neurons across different layers and channels. Our EfficientLIF-Net achieves comparable accuracy with the standard SNNs while bringing up to ~4.3× forward memory efficiency and ~21.9× backward memory efficiency for LIF neurons. We conduct experiments on various datasets including CIFAR10, CIFAR100, TinyImageNet, ImageNet-100, and N-Caltech101. Furthermore, we show that our approach also offers advantages on Human Activity Recognition (HAR) datasets, which heavily rely on temporal information. The code has been released at https://github.com/Intelligent-Computing-Lab-Yale/EfficientLIF-Net.

show abstract

“…Many works conduct multi-stage training, typically including an ANN pre-training process, to reduce the latency (i.e., the number of time steps) for the energy efficiency issue, while maintaining competitive performance [8,9,50,51]. The BPTT with SG method has achieved high performance with low latency on both static [21,24] and neuromorphic [16,37] datasets. However, those approaches need to backpropagate error signals through both temporal and spatial domains, thus suffering from high computational costs during training [14].…”

Section: Related Workmentioning

confidence: 99%

Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks

Meng¹,

Xiao²,

Shen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) are promising energyefficient models for neuromorphic computing. For training the non-differentiable SNN models, the backpropagation through time (BPTT) with surrogate gradients (SG) method has achieved high performance. However, this method suffers from considerable memory cost and training time during training. In this paper, we propose the Spatial Learning Through Time (SLTT) method that can achieve high performance while greatly improving training efficiency compared with BPTT. First, we show that the backpropagation of SNNs through the temporal domain contributes just a little to the final calculated gradients. Thus, we propose to ignore the unimportant routes in the computational graph during backpropagation. The proposed method reduces the number of scalar multiplications and achieves a small memory occupation that is independent of the total time steps. Furthermore, we propose a variant of SLTT, called SLTT-K, that allows backpropagation only at K time steps, then the required number of scalar multiplications is further reduced and is independent of the total time steps. Experiments on both static and neuromorphic datasets demonstrate superior training efficiency and performance of our SLTT. In particular, our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.

show abstract

Neuromorphic Data Augmentation for Training Spiking Neural Networks

Cited by 42 publications

References 65 publications

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

Sharing leaky-integrate-and-fire neurons for memory-efficient spiking neural networks

Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks

Contact Info

Product

Resources

About