Neural inference at the frontier of energy, space, and time

Abstract

Computing, since its inception, has been processor-centric, with memory separated from compute. Inspired by the organic brain and optimized for inorganic silicon, NorthPole is a neural inference architecture that blurs this boundary by eliminating off-chip memory, intertwining compute with memory on-chip, and appearing externally as an active memory chip. NorthPole is a low-precision, massively parallel, densely interconnected, energy-efficient, and spatial computing architecture with a co-optimized, high-utilization programming model. On the ResNet50 benchmark image classification network, relative to a graphics processing unit (GPU) that uses a comparable 12-nanometer technology process, NorthPole achieves a 25 times higher energy metric of frames per second (FPS) per watt, a 5 times higher space metric of FPS per transistor, and a 22 times lower time metric of latency. Similar results are reported for the Yolo-v4 detection network. NorthPole outperforms all prevalent architectures, even those that use more-advanced technology processes.

Controlling chaos using edge computing hardware

“…We optimize the overall system timing to minimize the latency, but the NG-RC model and control law is so simple that the evaluation time is two orders of magnitude shorter than the other latencies present in the controller, which are detailed Supplementary Note 7 . Another important aspect of our controller is that we use fixed-point arithmetic, which is matched to the digitized input data and reduces the required compute resources and power consumption 10 .…”

“…Regarding the power consumption, it can be likely be reduced to the sub-mW level using a more power-efficient FPGA on a custom circuit board without extraneous components. Because our algorithm only requires a small number of multiplications and additions, the model can be evaluated efficiently on the recent custom artificial-intelligence processors 10 , 11 or using a custom application-specific integrated circuit to achieve the lowest power consumption.…”

“…We implement our design using a FPGA on a commercial demonstration board to collect real-time data from the circuit, process the data using a firmware-based NG-RC, and apply the control perturbations. An FPGA has the advantage that it can perform parallel computations well matched to our ML algorithm and it has memory co-located with the processing logic gates, which is known to boost efficiency 10 , 11 . Furthermore, the ML algorithm is so simple and efficient that its execution time is much faster than other system processes and its power consumption is a tiny fraction of the overall power budget.…”

In Situ TEM Investigation of Thermally Induced Modifications of Cluster-Assembled Gold Films Undergoing Resistive Switching: Implications for Nanostructured Neuromorphic Devices

“…Current approaches relying on artificial neural networks (ANNs) have achieved exceptional performance in complex tasks like image recognition and text generation. , However, this comes at the cost of significantly increased energy consumption due to the inefficiency of computers based on the von Neumann paradigm . To mitigate the energy footprint of ANNs, there is a growing focus on developing computational strategies that mimic the capabilities of biological neural networks .…”

Resistive Switching Devices for Neuromorphic Computing: From Foundations to Chip Level Innovations

“…With immense research effort, CMOS-based neuromorphic chip designs like the TrueNorth [ 2 ], BrainScales [ 3 ], and Loihi 2 [ 4 ] have achieved break-through performance records. In 2023, IBM released the NorthPole chip [ 5 ], which is reported to be 25 times more energy efficient than corresponding graphical processing units (GPUs) in handling deep learning workloads. But the overall energy consumption and chip footprint still need to be considerably lowered.…”