The performance of a biologically plausible spiking neural network (SNN) largely depends on the model parameters and neural dynamics. This paper proposes a parameter optimization scheme for improving the performance of a biologically plausible SNN and a parallel on-FPGA online learning neuromorphic platform for the digital implementation based on two numerical methods, namely Euler and the 3rd-order Runge-Kutta (RK3) methods. The optimization scheme explores the impact of biological time constants on information transmission in the SNN and improves the convergence rate of the SNN on digit recognition with a suitable choice of the time constants. The parallel digital implementation leads to a significant speedup over software simulation on a general-purpose CPU. The parallel implementation with Euler method enables around 180× (20×) training (inference) speedup over a Pytorch-based SNN simulation on CPU. Moreover, compared with previous work, our parallel implementation shows more than 300× (240×) improvement on speed and 180×(250×) reduction on energy consumption for training (inference). In addition, due to the highorder accuracy, RK3 method is demonstrated to gain 2× training speedup over Euler method, which makes it suitable for online training in real-time applications.

To solve real-time challenges, neuromorphic systems generally require deep and complex network structures. Thus, it is crucial to search for effective solutions that can reduce network complexity, improve energy efficiency, and maintain high accuracy. To this end, we propose unsupervised pruning strategies that are focused on pruning neurons while training in spiking neural networks (SNNs) by utilizing network dynamics. The importance of neurons is determined by the fact that neurons that fire more spikes contribute more to network performance. Based on these criteria, we demonstrate that pruning with an adaptive spike count threshold provides a simple and effective approach that can reduce network size significantly and maintain high classification accuracy. The online adaptive pruning shows potential for developing energy-efficient training techniques due to less memory access and less weight-update computation. Furthermore, a parallel digital implementation scheme is proposed to implement spiking neural networks (SNNs) on field programmable gate array (FPGA). Notably, our proposed pruning strategies preserve the dense format of weight matrices, so the implementation architecture remains the same after network compression. The adaptive pruning strategy enables 2.3× reduction in memory size and 2.8× improvement on energy efficiency when 400 neurons are pruned from an 800-neuron network, while the loss of classification accuracy is 1.69%. And the best choice of pruning percentage depends on the trade-off among accuracy, memory, and energy. Therefore, this work offers a promising solution for effective network compression and energy-efficient hardware implementation of neuromorphic systems in real-time applications.

No abstract

Existing implementation methods of multi-port register files (MPo-RF) in FPGAs are not scalable enough to deal with the increased number of ports due to higher logic area and power. While the usage of dedicated block RAMs (BRAMs) limits the designer to use only single read and single write port, slice based approach causes large resource occupation and degrades design performance significantly. Similarly, the conventional multi-pumping (MPu) approaches are not efficient enough due to increased combinational delay and area of huge multiplexers. In this paper, we propose a new design which exploits the banking and replication of BRAMs with efficient shift register based multi-pumping (SR-MPu) approach. While increased port number causes internal frequency drops in conventional multiplexer based MPu approaches, it does not affect internal operating frequency of our SR-MPu methodology. Test results on Xilinx Virtex-5 XC5VLX110T FPGA show that our 32-bit 12-read & 6-write (12R&6W) RF can operate internally up to 429 Mhz while 64-bit version up to 408 Mhz. The speed of our RF is independent from MPu factor and occupies lower logic resources up to 47% when compared with other design methods. In terms of energy consumption, our RF design saves energy up to 26% according to the Xilinx Power Analyzer (XPA) results.

To tackle real-world challenges, deep and complex neural networks are generally used with a massive number of parameters, which require large memory size, extensive computational operations, and high energy consumption in neuromorphic hardware systems. In this work, we propose an unsupervised online adaptive weight pruning method that dynamically removes non-critical weights from a spiking neural network (SNN) to reduce network complexity and improve energy efficiency. The adaptive pruning method explores neural dynamics and firing activity of SNNs and adapts the pruning threshold over time and neurons during training. The proposed adaptation scheme allows the network to effectively identify critical weights associated with each neuron by changing the pruning threshold dynamically over time and neurons. It balances the connection strength of neurons with the previous layer with adaptive thresholds and prevents weak neurons from failure after pruning. We also evaluated improvement in the energy efficiency of SNNs with our method by computing synaptic operations (SOPs). Simulation results and detailed analyses have revealed that applying adaptation in the pruning threshold can significantly improve network performance and reduce the number of SOPs. The pruned SNN with 800 excitatory neurons can achieve a 30% reduction in SOPs during training and a 55% reduction during inference, with only 0.44% accuracy loss on MNIST dataset. Compared with a previously reported online soft pruning method, the proposed adaptive pruning method shows 3.33% higher classification accuracy and 67% more reduction in SOPs. The effectiveness of our method was confirmed on different datasets and for different network sizes. Our evaluation showed that the implementation overhead of the adaptive method regarding speed, area, and energy is negligible in the network. Therefore, this work offers a promising solution for effective network compression and building highly energy-efficient neuromorphic systems in real-time applications.