Hadamard product-based in-memory computing design for floating point neural network training

Abstract

Deep neural networks (DNNs) are one of the key fields of machine learning. It requires considerable computational resources for cognitive tasks. As a novel technology to perform computing inside/near memory units, in-memory computing (IMC) significantly improves computing efficiency by reducing the need for repetitive data transfer between the processing and memory units. However, prior IMC designs mainly focus on the acceleration for DNN inference. DNN training with the IMC hardware has rarely been proposed. The challenges lie in the requirement of DNN training for high precision (e.g. floating point) and various operations of tensors (e.g. inner and outer products). These challenges call for the IMC design with new features. This paper proposes a novel Hadamard product-based IMC design for floating point DNN training. Our design consists of multiple compartments, which are the basic units for the matrix element-wise processing. We also develop BFloat16 post-processing circuits and fused adder trees, laying the foundation for IMC floating point processing. Based on the proposed circuit scheme, we reformulate the back-propagation training algorithm for the convenience and efficiency of the IMC execution. The proposed design is implemented with commercial 28nm technology process design kits and benchmarked with widely used neural networks. We model the influence of the circuit structural design parameters and provide an analysis framework for design space exploration. Our simulation validates that MobileNet training with the proposed IMC scheme saves 91.2% in energy and 13.9% in time versus the same task with NVIDIA GTX 3060 GPU. The proposed IMC design has a data density of 769.2Kb/mm2 with the floating point processing circuits included, showing a 3.5× improvement than the prior floating point IMC designs.

Editorial: Focus issue on in-memory computing

“…In Fan et al [1], the authors developed efficient in-memory circuits to perform both floating-point vector-matrix multiplication and vector Hadamard product (element-wise product) operations that are necessary for neural network training. The proposed IMC design, validated on 28 nm technology design kit, enables both neural network inference and training at the edge, and improves the data density by 3.5 × compared to previous designs with negligible precision loss compared to the ideal BFloat16 computing.…”