Fast Convergence of Competitive Spiking Neural Networks with Sample-Based Weight Initialization

Cachi, Paolo G.; Ventura, Sebastián; Cios, Krzysztof J.

doi:10.1007/978-3-030-50153-2_57

Cited by 3 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiments are divided into five parts. First, we analyze learning performance of CRBA (in terms of accuracy, and firing threshold and weight evolution) and compare it with performance of CSNN's implementation that uses sample-based initialization of weights (Cachi et al, 2020) (details are provided in the Supplementary Material). Second, we analyze usage of weights and firing thresholds found by CRBA to initialize CSNN (CSNN+CRBA).…”

Section: Resultsmentioning

confidence: 99%

“…To initialize the weights, W, we use sample-based initialization that was shown to reduce the number of samples needed for learning (Cachi et al, 2020).…”

Section: Parameter Selectionmentioning

confidence: 99%

“…The learning phase uses input data to, first, tune the competitive neurons in a completely unsupervised way (i.e., find weights and firing thresholds using Algorithm 1) and, second, assign labels to the already tuned neurons based on the per-class neuron activity (using labeled samples). In the testing phase, the already labeled neurons are used to predict the labels for new unseen samples using a maximum-voting scheme (Querlioz et al, 2011;Hazan et al, 2018;Cachi et al, 2020), or using an add-on classifier, such as a fully connected feedfoward neural network (Demin and Nekhaev, 2018;Kheradpisheh et al, 2018;Krotov and Hopfield, 2019).…”

Section: Usagementioning

confidence: 99%

“…A Competitive Spiking Neural Network (CSNN), is a two-layer feedforward spiking network with lateral inhibitory connections (Querlioz et al, 2011;Cachi et al, 2020), that uses spiking neurons with local Konorski/Hebb learning rule to implement a dynamic temporal network that exhibits properties often missing in deep learning models. They are pattern selectivity (spiking neurons learn to detect specific input patterns) (Masquelier and Thorpe, 2007;Nessler et al, 2009;Lobov et al, 2020), short-/long-term memory (spiking neurons use self-regulatory mechanism that processes information in different time scales) (Ermentrout, 1998;Brette and Gerstner, 2005;Pfister and Gerstner, 2006;Zenke et al, 2015), synaptic plasticity (based on local learning first observed by Konorski and then by Hebb) (Konorski, 1948;Hebb, 1949), modularity (spiking neurons operate and learn independently) (Zylberberg et al, 2011;, adaptability, and continuous learning (Brette and Gerstner, 2005;Wysoski et al, 2008).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

CRBA: A Competitive Rate-Based Algorithm Based on Competitive Spiking Neural Networks

Cachi

Ventura

Cios

2021

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

In this paper we present a Competitive Rate-Based Algorithm (CRBA) that approximates operation of a Competitive Spiking Neural Network (CSNN). CRBA is based on modeling of the competition between neurons during a sample presentation, which can be reduced to ranking of the neurons based on a dot product operation and the use of a discrete Expectation Maximization algorithm; the latter is equivalent to the spike time-dependent plasticity rule. CRBA's performance is compared with that of CSNN on the MNIST and Fashion-MNIST datasets. The results show that CRBA performs on par with CSNN, while using three orders of magnitude less computational time. Importantly, we show that the weights and firing thresholds learned by CRBA can be used to initialize CSNN's parameters that results in its much more efficient operation.

show abstract

Section: Resultsmentioning

confidence: 99%

“…To initialize the weights, W, we use sample-based initialization that was shown to reduce the number of samples needed for learning (Cachi et al, 2020).…”

Section: Parameter Selectionmentioning

confidence: 99%

Section: Usagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CRBA: A Competitive Rate-Based Algorithm Based on Competitive Spiking Neural Networks

Cachi

Ventura

Cios

2021

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Data-driven initialization techniques [154,[167][168][169][170][171][172] are proposed in which the weights are derived from the training data. Gan et al [167] propose to initialize the filter weights of the CNN using principal component analysis (PCA) [173].…”

Section: Network Initializationmentioning

confidence: 99%

Transferable and curious AI in sequential recommendation

Zhang¹

View full text Add to dashboard Cite

Reaching the end of my PhD, I would like to express my sincerest gratitude to all those who have guided me, helped me, and encouraged me during this bittersweet journey. First and foremost, I am tremendously grateful to my supervisor, Prof. Chunyan Miao for the kind support and mentorship throughout my PhD. Her kind support and keen insight were key factors in making my research possible. I am very glad to have the chance to collaborate with Prof. Boyang Li and Dr. Yong Liu. During the discussions with them, I learned how to do research with critical thinking and a rigorous attitude. I would like to thank my committee members, Prof. Han Yu and Prof. Lihui Chen, for their guidance. I am also grateful for the teaching assistant opportunities provided by Dr. Zhiqi Shen.

show abstract