Hardware-Amenable Structural Learning for Spike-Based Pattern Classification Using a Simple Model of Active Dendrites

Abstract

This letter presents a spike-based model that employs neurons with functionally distinct dendritic compartments for classifying high-dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron the capacity to perform a large number of input-output mappings. The model uses sparse synaptic connectivity, where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin-enhancing learning algorithm inspired by the mechanism of structural plasticity in biological neurons. The learning algorithm groups correlated synaptic inputs on the same dendritic branch. Since the learning results in modified connection patterns, it can be incorporated into current event-based neuromorphic systems with little overhead. This work also presents a branch-specific spike-based version of this structural plasticity rule. The proposed model is evaluated on benchmark binary classification problems, and its performance is compared against that achieved using support vector machine and extreme learning machine techniques. Our proposed method attains comparable performance while using 10% to 50% less in computational resource than the other reported techniques.

Structural Plasticity Denoises Responses and Improves Learning Speed

“…Hussain et al implemented a model which clusters correlated synapses on the same dendritic branch with a hardware-friendly learning rule (Hussain et al, 2015). The proposed model attains comparable performance to Support Vector Machines and Extreme Learning Machines on binary classification benchmarks while using less computational resources.…”

Hardware architecture for large parallel array of Random Feature Extractors applied to image recognition

“…Conventionally, at least D × L random weights are needed for the random projection operation in the first layer of ELM to get the hidden layer matrix H. However if the number of implemented hidden layer neurons is N (N < L), the hardware can only provide a D × N random projection matrix W comprising weights w ij (i = 1, 2, · · · , D and j = 1, 2, · · · , N). However, noting that we have a total of D × N random numbers on the chip, we can borrow concepts from combinatorics based learning [20][21][22] to realize that the total number N w of D-dimensional weight vectors we can make is given by:…”

An Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks

“…This architecture, which we refer to as Winner-Take-All employing Neurons with NonLinear Dendrites (WTA-NNLD), uses a novel branch-specific Spike Timing Dependent Plasticity based Network Rewiring (STDP-NRW) learning rule for its training. We have earlier presented [23] a branch-specific STDP rule for batch learning of a supervised classifier constructed of NNLDs. The primary differences of our current approach with [23] are:…”

“…We have earlier presented [23] a branch-specific STDP rule for batch learning of a supervised classifier constructed of NNLDs. The primary differences of our current approach with [23] are:…”