StereoSpike: Depth Learning with a Spiking Neural Network

Rançon, Ulysse; Cuadrado-Anibarro, Javier; Cottereau, Benoit R.; Masquelier, Timothée

doi:10.48550/arxiv.2109.13751

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The increasing number of adoptions by the community is symbolic of the success of a framework. Since being open-sourced in December 2019, SpikingJelly has been widely used in many spiking deep learning studies, including adversarial attack ( 100 , 101 ), ANN2SNN ( 95 , 102 – 106 ), attention mechanisms ( 107 , 108 ), depth estimation from DVS data ( 69 , 109 ), development of innovative materials ( 110 ), emotion recognition ( 111 ), energy estimation ( 112 ), event-based video reconstruction ( 113 ), fault diagnosis ( 114 ), hardware design ( 115 – 117 ), network structure improvements ( 60 , 61 , 118 – 121 ), spiking neuron improvements ( 56 , 122 – 127 ), training method improvements ( 128 – 138 ), medical diagnosis ( 139 , 140 ), network pruning ( 141 – 145 ), neural architecture search ( 146 , 147 ), neuromorphic data augmentation ( 148 ), natural language processing ( 149 ), object detection/tracking for DVS/frame data ( 65 , 66 , 150 ), odor recognition ( 151 ), optical flow estimation with DVS data ( 152 ), reinforcement learning for controlling ( 153 – 155 ), and semantic communication ( 156 ). Figure 1F shows parts of these adoptions.…”

Section: Resultsmentioning

confidence: 99%

“…SNNs trained by the surrogate gradient method achieve high performance on complex datasets such as the Canadian Institute for Advanced Research (CIFAR) dataset (54), the Dynamic Vision Sensor (DVS) Gesture dataset (55) and the challenging Im-ageNet dataset (19) using only a few simulation time steps (56)(57)(58)(59)(60)(61), while SNNs converted from ANNs attain almost the same accuracy as that of the original ANNs on the ImageNet dataset with dozens of simulation time steps (51,62,63). Because of the rapid progress achieved by deep learning methods, the applications of SNNs have been expanded beyond classification to other tasks including object detection (64)(65)(66), object segmentation (67,68), depth estimation (69), and optical flow estimation (70). The boom exhibited by the research community indicates that spiking deep learning has become a promising research hotspot.…”

Section: Emerging Spiking Deep Learning Methodsmentioning

confidence: 99%

Section: Adoptions By the Communitymentioning

confidence: 99%

See 2 more Smart Citations

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Fang,

Chen,

Ding

et al. 2023

Sci. Adv.

Self Cite

View full text Add to dashboard Cite

Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties. As the emerging spiking deep learning paradigm attracts increasing interest, traditional programming frameworks cannot meet the demands of the automatic differentiation, parallel computation acceleration, and high integration of processing neuromorphic datasets and deployment. In this work, we present the SpikingJelly framework to address the aforementioned dilemma. We contribute a full-stack toolkit for preprocessing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips. Compared to existing methods, the training of deep SNNs can be accelerated 11×, and the superior extensibility and flexibility of SpikingJelly enable users to accelerate custom models at low costs through multilevel inheritance and semiautomatic code generation. SpikingJelly paves the way for synthesizing truly energy-efficient SNN-based machine intelligence systems, which will enrich the ecology of neuromorphic computing.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Emerging Spiking Deep Learning Methodsmentioning

confidence: 99%

Section: Adoptions By the Communitymentioning

confidence: 99%

See 1 more Smart Citation

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Fang,

Chen,

Ding

et al. 2023

Sci. Adv.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In Hagenaars et al (2021) and Kosta and Roy (2022), an event-based optical flow estimation method was presented. In StereoSpike (Rançon et al, 2021) a depth estimation method was provided. SuperFast (Gao et al, 2022) leveraged an SNN and an event camera to present an event-enhanced high-speed video frame interpolation method.…”

Section: Cooperating With Neuromorphic Camerasmentioning

confidence: 99%

Direct learning-based deep spiking neural networks: a review

Guo

Huang

2023

Front. Neurosci.

View full text Add to dashboard Cite

The spiking neural network (SNN), as a promising brain-inspired computational model with binary spike information transmission mechanism, rich spatially-temporal dynamics, and event-driven characteristics, has received extensive attention. However, its intricately discontinuous spike mechanism brings difficulty to the optimization of the deep SNN. Since the surrogate gradient method can greatly mitigate the optimization difficulty and shows great potential in directly training deep SNNs, a variety of direct learning-based deep SNN works have been proposed and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods. In addition, we also divide these categorizations into finer granularities further to better organize and introduce them. Finally, the challenges and trends that may be faced in future research are prospected.

show abstract

StereoSpike: Depth Learning with a Spiking Neural Network

Cited by 2 publications

References 0 publications

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Direct learning-based deep spiking neural networks: a review

Contact Info

Product

Resources

About