Synthesis of neural networks for spatio-temporal spike pattern recognition and processing

Abstract

The advent of large scale neural computational platforms has highlighted the lack of algorithms for synthesis of neural structures to perform predefined cognitive tasks. The Neural Engineering Framework (NEF) offers one such synthesis, but it is most effective for a spike rate representation of neural information, and it requires a large number of neurons to implement simple functions. We describe a neural network synthesis method that generates synaptic connectivity for neurons which process time-encoded neural signals, and which makes very sparse use of neurons. The method allows the user to specify—arbitrarily—neuronal characteristics such as axonal and dendritic delays, and synaptic transfer functions, and then solves for the optimal input-output relationship using computed dendritic weights. The method may be used for batch or online learning and has an extremely fast optimization process. We demonstrate its use in generating a network to recognize speech which is sparsely encoded as spike times.

A neuromorphic hardware framework based on population coding

“…4C). The learning capability of an LSHDI network depends on the number of hidden layer neurons [8]. As evident from Fig.…”

“…LSHDI networks are represented as having three layers of neurons -input, hidden and output layers, in a feedforward structure [8]. These networks differ from similar neural network architectures in several ways -(i) the hidden layer is usually much larger than the input layer, (ii) the connections between the input layer and the hidden layer are randomly generated, and (iii) the connections do not change during the network training.…”

“…Input data points, which are not linearly separable in their current space, allow a linear hyperplane in the higher dimensional space that approximates a desired function as a regression solution, or represents a classification boundary for the input-output relationship. The output layer neurons need therefore compute only a linearly weighted sum of the hidden layer values in order to solve the problem, hence the name Linear Solutions of Higher Dimensional Interlayers (LSHDI) [8]. These linear weights are determined analytically by calculating the product of the pseudoinverse of the hidden layer activations with the desired output values [26].…”

“…To our knowledge, our work is the first of its kind to present measurement results of the TAB, which employs random device mismatch to implement a neural network. The TAB implementation is based on the LSHDI (Linear Solutions of Higher Dimensional Interlayers) framework [8], which will be explained in detail in section III. We present the measurements of the building blocks of our TAB hardware implementation in section IV, algorithm for offline learning in section V, mathematical proof for importance of heterogeneity of tuning curves in section VI, and capability of the TAB to learn various regression tasks in section VII.…”

ELM solutions for event-based systems

“…We have used an early variant of the SKIM method to tackle this problem, and the successful result is reported elsewhere [22]. For the purposes of illustrating the method here, we will use a generic problem that has features of many different types of application.…”