Towards Neuromorphic Learning Machines using Emerging Memory Devices with Brain-like Energy Efficiency

Abstract

The ongoing revolution in Deep Learning is redefining the nature of computing that is driven by the increasing amount of pattern classification and cognitive tasks. Specialized digital hardware for deep learning still holds its predominance due to the flexibility offered by the software implementation and maturity of algorithms. However, it is being increasingly desired that cognitive computing occurs at the edge, i.e. on hand-held devices that are energy constrained, which is energy prohibitive when employing digital von Neumann architectures. Recent explorations in digital neuromorphic hardware have shown promise, but offer low neurosynaptic density needed for scaling to applications such as intelligent cognitive assistants (ICA). Large-scale integration of nanoscale emerging memory devices with Complementary Metal Oxide Semiconductor (CMOS) mixed-signal integrated circuits can herald a new generation of Neuromorphic computers that will transcend the von Neumann bottleneck for cognitive computing tasks. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures promise machine learning capability at chip-scale form factor, and several orders of magnitude improvement in energy efficiency. Practical demonstration of such architectures has been limited as performance of emerging memory devices falls short of the expected behavior from the idealized memristor-based analog synapses, or weights, and novel machine learning algorithms are needed to take advantage of the device behavior. In this work, we review the challenges involved and present a pathway to realize ultra-low-power mixed-signal NeuSoC, from device arrays and circuits to spike-based deep learning algorithms, with ‘brain-like’ energy-efficiency.

Implementing an Insect Brain Computational Circuit Using III–V Nanowire Components in a Single Shared Waveguide Optical Network

“…More detailed studies of the energy consumption of the neural system in the brain have been put forward, however, estimates end up in a similar range. 56 Exactly how the brain spends this energy is a matter of debate, but it has been estimated that around 70% is used for interneuron communication. 57 Using CMOS solutions particularly optimized toward neural networks, efficiencies in the range of 10 –11 J per operation have been achieved.…”

A Crossbar Array of Analog-Digital-Hybrid Volatile Memory Synapse Cells for Energy-Efficient On-Chip Learning

“…So an in-memory computing system like a crossbar array of analog synaptic devices has been considered as a faster and energyefficient alternative to traditional computing systems, with memory-computing separation, for executing these algorithms [2]- [7]. While such a crossbar array can be used for inference (forward computation with pre-trained NN), recently the option of training the NN in the array itself has also been explored -also known as on-chip learning -owing to the advantages it offers for edge devices [4], [8], [9].…”