A Sparse Coding Neural Network ASIC With On-Chip Learning for Feature Extraction and Encoding

Abstract

Hardware-based computer vision accelerators will be an essential part of future mobile devices to meet the low power and real-time processing requirement. To realize a high energy efficiency and high throughput, the accelerator architecture can be massively parallelized and tailored to vision processing, which is an advantage over software-based solutions and general-purpose hardware. In this work, we present an ASIC that is designed to learn and extract features from images and videos. The ASIC contains 256 leaky integrate-and-fire neurons connected in a scalable two-layer network of 8 8 grids linked in a 4-stage ring. Sparse neuron activation and the relatively small grid keep the spike collision probability low to save access arbitration. The weight memory is divided into core memory and auxiliary memory, such that the auxiliary memory is only powered on for learning to save inference power. High-throughput inference is accomplished by the parallel operation of neurons. Efficient learning is implemented by passing parameter update messages, which is further simplified by an approximation technique. A 3.06 mm 65 nm CMOS ASIC test chip is designed to achieve a maximum inference throughput of 1.24Gpixel/s at 1.0 V and 310 MHz, and on-chip learning can be completed in seconds. To improve the power consumption and energy efficiency, core memory supply voltage can be reduced to 440 mV to take advantage of the error resilience of the algorithm, reducing the inference power to 6.67 mW for a 140 Mpixel/s throughput at 35 MHz.Index Terms-Feature extraction, hardware acceleration, neural network architecture, sparse coding, sparse and independent local network.

A Self-Organizing Short-Term Dynamical Memory Network

“…This same setup could be utilized in RNNs to make them more biologically realistic. This would let us better understand how the brain learns, and could lead to novel biomimetic technologies: prior work on biologically realistic machine learning algorithms has led to hardware devices that use on-chip learning (Knag et al, 2015;Zylberberg et al, 2011). Synaptically local updates do not have to be coordinated over all parts of the chip, enabling simpler and more efficient hardware implementations.…”

Fast and Accurate Sparse Coding of Visual Stimuli With a Simple, Ultralow-Energy Spiking Architecture

“…Not only does using fewer outputs result in a worse encoding of the input, but due to the O(N 2 ) scaling properties of these networks, this also favors the power and throughput figures in [27]. As such, subsequent results in this work will be compared with Knag et al, as their chip performs a comparable amount of work to ours [25]. Like the LCA, SAILnet uses a direct inhibitory weight between each pair of output neurons, yielding a scaling complexity of O(N 2 ).…”

“…ASICs using this architecture have been studied, with a substantial reduction in power compared to the approach presented by Shapero et al [16]. Knag et al [25] were capable of using the SAILnet architecture to process images using only 48 pJ/input for their inference logic with a throughput of 0.55 MOps/s, or using 176 pJ/input with a throughput of 4.8 MOps/s, 120 × as fast as Shapero et al [16]. Their design was CMOS-based, and utilized a decreased resolution for weight storage: 4 bits per excitatory or inhibitory weight.…”

“…They tested higher clock speeds, optimized the design, and generated less detailed output, resulting in a throughput of 9.9 MOps/s at an energy efficiency of 26.4 pJ/input [27]. However, these numbers benefit from their usage of only 256 output neurons to represent 1024 inputs, whereas Knag et al used 256 output neurons to represent only 256 inputs [25], [27]. Not only does using fewer outputs result in a worse encoding of the input, but due to the O(N 2 ) scaling properties of these networks, this also favors the power and throughput figures in [27].…”

MorphIC: A 65-nm 738k-Synapse/mm$^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning

“…1(b). Spike-based online learning is an active research area, both in the development of new rules for high-accuracy learning in multi-layer networks (e.g., [9]- [12]) and in the demonstration of silicon implementations in applications such arXiv:1904.08513v2 [cs.NE] 16 Jul 2019 as unsupervised learning for image denoising and reconstruction [13], [14]. However, these approaches currently rely on multi-bit weights.…”