SpiLinC: Spiking Liquid-Ensemble Computing for Unsupervised Speech and Image Recognition

Abstract

In this work, we propose a Spiking Neural Network (SNN) consisting of input neurons sparsely connected by plastic synapses to a randomly interlinked liquid, referred to as Liquid-SNN, for unsupervised speech and image recognition. We adapt the strength of the synapses interconnecting the input and liquid using Spike Timing Dependent Plasticity (STDP), which enables the neurons to self-learn a general representation of unique classes of input patterns. The presented unsupervised learning methodology makes it possible to infer the class of a test input directly using the liquid neuronal spiking activity. This is in contrast to standard Liquid State Machines (LSMs) that have fixed synaptic connections between the input and liquid followed by a readout layer (trained in a supervised manner) to extract the liquid states and infer the class of the input patterns. Moreover, the utility of LSMs has primarily been demonstrated for speech recognition. We find that training such LSMs is challenging for complex pattern recognition tasks because of the information loss incurred by using fixed input to liquid synaptic connections. We show that our Liquid-SNN is capable of efficiently recognizing both speech and image patterns by learning the rich temporal information contained in the respective input patterns. However, the need to enlarge the liquid for improving the accuracy introduces scalability challenges and training inefficiencies. We propose SpiLinC that is composed of an ensemble of multiple liquids operating in parallel. We use a “divide and learn” strategy for SpiLinC, where each liquid is trained on a unique segment of the input patterns that causes the neurons to self-learn distinctive input features. SpiLinC effectively recognizes a test pattern by combining the spiking activity of the constituent liquids, each of which identifies characteristic input features. As a result, SpiLinC offers competitive classification accuracy compared to the Liquid-SNN with added sparsity in synaptic connectivity and faster training convergence, both of which lead to improved energy efficiency in neuromorphic hardware implementations. We validate the efficacy of the proposed Liquid-SNN and SpiLinC on the entire digit subset of the TI46 speech corpus and handwritten digits from the MNIST dataset.

Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines

“…In contrast, we analyze the effects of dividing a large liquid into multiple smaller units, while leaving the total number of neurons the same. Research (Srinivasan et al, 2018) also shows that multiple liquids perform better than a single liquid, at higher number of neurons. The input to liquid connections in Srinivasan et al (2018) were trained in an unsupervised manner.…”

“…Research (Srinivasan et al, 2018) also shows that multiple liquids perform better than a single liquid, at higher number of neurons. The input to liquid connections in Srinivasan et al (2018) were trained in an unsupervised manner. Also note that each liquid was fed with distinct parts of an input, and hence is different from this work.…”

“…(Kötter, 2012) and using enhanced learning rules (Roy et al, 2013). LSMs have been used for a variety of applications including robot control (Urbain et al, 2017), sequence generation (Panda and Roy, 2017), decoding actual brain activity (Nikolić et al, 2009), action recognition (Panda and Srinivasa, 2018), speech recognition (Maass et al, 2002; Verstraeten et al, 2005; Goodman and Ventura, 2006; Zhang et al, 2015; Wu et al, 2018; Zhang and Li, 2019), and image recognition (Grzyb et al, 2009; Wang and Li, 2016; Srinivasan et al, 2018; Zhang and Li, 2019).…”

Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures

“…Most of the classification performances available in literature for SNNs are for MNIST and CIFAR-10 datasets. The popular methods for SNN training are 'Spike Time Dependent Plasticity (STDP)' based unsupervised learning [7,49,3,42,43] and 'spike-based backpropagation' based supervised learning [24,16,48,30,29]. There are a few works [45,17,46,22] which tried to combine the two approaches to get the best of both worlds.…”

“…STDP-trained two-layer network (consisting of 6400 output neurons) has been shown to achieve 95% classification accuracy on MNIST dataset. However, shallow network structure limits the expressive power of neural network [7,49,3,42,43] and suffers from scalability issues as the classification performance easily saturates. Layer-wise STDP learning [17,23] has shown the capabilities of efficient feature extraction on multi-layer convolutional SNNs.…”