Approximate computing for spiking neural networks

Sen, Sanchari; Venkataramani, Swagath; Raghunathan, Anand

doi:10.23919/date.2017.7926981

Cited by 47 publications

(31 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What is interesting with weight quantization is that the designer can realize a trade-off between the accuracy of the SNN application against energy and area requirements of the neural networks. Approximate computing can also be achieved at the neuron level, where insignificant units are deactivated to reduce computation cost of evaluating SNNs [163].…”

Section: Exploit Fault Tolerance Of Neural Networkmentioning

confidence: 99%

Spiking Neural Networks Hardware Implementations and Challenges

Bouvier

Valentian

Mesquida

et al. 2019

J. Emerg. Technol. Comput. Syst.

197

View full text Add to dashboard Cite

Neuromorphic computing is henceforth a major research field for both academic and industrial actors. As opposed to Von Neumann machines, brain-inspired processors aim at bringing closer the memory and the computational elements to efficiently evaluate machine-learning algorithms. Recently, Spiking Neural Networks, a generation of cognitive algorithms employing computational primitives mimicking neuron and synapse operational principles, have become an important part of deep learning. They are expected to improve the computational performance and efficiency of neural networks, but are best suited for hardware able to support their temporal dynamics. In this survey, we present the state of the art of hardware implementations of spiking neural networks and the current trends in algorithm elaboration from model selection to training mechanisms. The scope of existing solutions is extensive; we thus present the general framework and study on a case-by-case basis the relevant particularities. We describe the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level and discuss their related advantages and challenges.

show abstract

Section: Exploit Fault Tolerance Of Neural Networkmentioning

confidence: 99%

Spiking Neural Networks Hardware Implementations and Challenges

Bouvier

Valentian

Mesquida

et al. 2019

J. Emerg. Technol. Comput. Syst.

197

View full text Add to dashboard Cite

show abstract

“…Studies have revealed that the computing accuracy of synapses and neurons influences the final results of computations. Therefore, SNNs can be employed to solve problems for which only approximate answers are required, thereby conserving resources [31]. Scholars have also discovered that the computing costs of neural networks can be reduced using compression and sparse methods [32].…”

Section: Current Development Of Neurocomputing Technologymentioning

confidence: 99%

Using Patent Technology Networks to Observe Neurocomputing Technology Hotspots and Development Trends

Chang¹,

Fan²

2020

Sustainability

View full text Add to dashboard Cite

In recent years, development in the fields of big data and artificial intelligence has given rise to interest among scholars in neurocomputing-related applications. Neurocomputing has relatively widespread applications because it is a critical technology in numerous fields. However, most studies on neurocomputing have focused on improving related algorithms or application fields; they have failed to highlight the main technology hotspots and development trends from a comprehensive viewpoint. To fill the research gap, this study adopts a new viewpoint and employs technological fields as its main subject. Neurocomputing patents are subjected to network analysis to construct a neurocomputing technology hotspot. The results reveal that the neurocomputing technology hotspots are algorithms, methods or devices for reading or recognizing printed or written characters or patterns, and digital storage characterized by the use of particular electric or magnetic storage elements. Furthermore, the technology hotspots are discovered to not be clustered around particular fields but, rather, are multidisciplinary. The applications that combine neurocomputing with digital storage are currently undergoing the most extensive development. Finally, patentee analysis reveal that neurocomputing technology is mainly being developed by information technology corporations, thereby indicating the market development potential of neurocomputing technology. This study constructs a technology hotspot network model to elucidate the trend in development of neurocomputing technology, and the findings may serve as a reference for industries planning to promote emerging technologies.

show abstract

“…Approximate computing is well known in the area of signal processing and neural network hardware, but has seen limited application to spiking networks. One example is [15], where neurons are progressively trimmed from evaluation as time progresses. Again, our approach is orthogonal to this, and could be used to further reduce computations even for those neurons that are being evaluated.…”

Section: Approximate and Emerging Technologiesmentioning

confidence: 99%

Probabilistic spike propagation for FPGA implementation of spiking neural networks

Nallathambi¹,

Chandrachoodan²

2020

Preprint

View full text Add to dashboard Cite

Evaluation of spiking neural networks requires fetching a large number of synaptic weights to update postsynaptic neurons. This limits parallelism and becomes a bottleneck for hardware.We present an approach for spike propagation based on a probabilistic interpretation of weights, thus reducing memory accesses and updates. We study the effects of introducing randomness into the spike processing, and show on benchmark networks that this can be done with minimal impact on the recognition accuracy.We present an architecture and the trade-offs in accuracy on fully connected and convolutional networks for the MNIST and CIFAR10 datasets on the Xilinx Zynq platform. IntroductionSpiking neural networks are often referred to as the third generation of neural network models [10], and have several characteristics that make them attractive from the viewpoint of hardware design. They follow an event-driven model of computation, where the work done (hence energy consumed) can be made proportional to the number of spike events, and do not require the arrays of multiply-accumulate (MAC) operations that characterize conventional artificial neural networks (including convolutional networks, referred to here as ANNs) This makes ANNs well-suited to parallel implementation on architectures such as GPUs.In contrast, spiking neural networks may require other types of computations to determine whether a neuron is to fire or not. Hardware architectures for spiking networks (eg. [2, 7, 4, 11]) therefore differ considerably from those for regular ANNs, and focus more on features that enable efficient event-driven computation. This usually requires the network to be trained specifically for the target architecture (due to restrictions on permitted connections or weights), and it is not efficient to take a network trained for one architecture and directly run it on another.Despite being event driven, spiking networks still require a large number of memory accesses, primarily for two purposes [11]: determining the recipient neurons of a spike, and fetching weights of the corresponding synapses. Recent data (eg. [8]) indicates that fetching data from memory (especially off-chip) is much more expensive in energy than arithmetic computations. For reasonably sized networks, the neuron weight and index information becomes too much to store on-chip -the resulting off-chip memory accesses thus end up dominating the energy consumed for the computation, and can also lead to increased latency.

show abstract

Approximate computing for spiking neural networks

Cited by 47 publications

References 22 publications

Spiking Neural Networks Hardware Implementations and Challenges

Spiking Neural Networks Hardware Implementations and Challenges

Using Patent Technology Networks to Observe Neurocomputing Technology Hotspots and Development Trends

Probabilistic spike propagation for FPGA implementation of spiking neural networks

Contact Info

Product

Resources

About