A 28nm 29.2TFLOPS/W BF16 and 36.5TOPS/W INT8 Reconfigurable Digital CIM Processor with Unified FP/INT Pipeline and Bitwise In-Memory Booth Multiplication for Cloud Deep Learning Acceleration

Tu, Fengbin; Wang, Yiqi; Wu, Zihan; Liang, Ling; Ding, Yufei; Kim, Bongjin; Liu, Leibo; Wei, Shaojun; Xie, Yuan; Yin, Shouyi

doi:10.1109/isscc42614.2022.9731762

Cited by 63 publications

(14 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet most existing SRAM-based IMC structures have low data density 8 . The data density in [41,42] is lower than 0.3Mb mm −2 . This inevitably causes more off-chip memory communication during training, and can hardly scale out for large-scale training tasks at low cost.…”

Section: Introductionmentioning

confidence: 80%

“…Recently, there are several works employing floating-point formats for IMC designs: Tu et al [41] developed a digital processor handling GeMM between integers and FP numbers, reaching 14 TFLOPS/W efficiency computing GeMM with BFloat16 format. Lee et al [42] designed an SRAM-based IMC circuit processing GeMM between BFloat16, which separates exponent and fraction storage with the best computing efficiency at 1.43 TFLOPS W −1 ; Lee et al [43] give a DRAM-based near-memory computing design with high throughput at 1 TFLOPS per chip.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hadamard product-based in-memory computing design for floating point neural network training

Fan

Tao

et al. 2023

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Deep neural networks (DNNs) are one of the key fields of machine learning. It requires considerable computational resources for cognitive tasks. As a novel technology to perform computing inside/near memory units, in-memory computing (IMC) significantly improves computing efficiency by reducing the need for repetitive data transfer between the processing and memory units. However, prior IMC designs mainly focus on the acceleration for DNN inference. DNN training with the IMC hardware has rarely been proposed. The challenges lie in the requirement of DNN training for high precision (e.g. floating point) and various operations of tensors (e.g. inner and outer products). These challenges call for the IMC design with new features. This paper proposes a novel Hadamard product-based IMC design for floating point DNN training. Our design consists of multiple compartments, which are the basic units for the matrix element-wise processing. We also develop BFloat16 post-processing circuits and fused adder trees, laying the foundation for IMC floating point processing. Based on the proposed circuit scheme, we reformulate the back-propagation training algorithm for the convenience and efficiency of the IMC execution. The proposed design is implemented with commercial 28nm technology process design kits and benchmarked with widely used neural networks. We model the influence of the circuit structural design parameters and provide an analysis framework for design space exploration. Our simulation validates that MobileNet training with the proposed IMC scheme saves 91.2% in energy and 13.9% in time versus the same task with NVIDIA GTX 3060 GPU. The proposed IMC design has a data density of 769.2Kb/mm2 with the floating point processing circuits included, showing a 3.5× improvement than the prior floating point IMC designs.

show abstract

Section: Introductionmentioning

confidence: 80%

Section: Introductionmentioning

confidence: 99%

Hadamard product-based in-memory computing design for floating point neural network training

Fan

Tao

et al. 2023

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

show abstract

“…In general, the fully digital approach is exceptionally robust to various nonidealities, such as device variability, drift, noise, or IR drop, and it can have higher reconfigurability [134]- [136]. However, because of the accumulation through counting, the parallelism of the computation is limited to just one Fig.…”

Section: A Fully Digital Circuitsmentioning

confidence: 99%

In-Memory Computing for Machine Learning and Deep Learning

Lepri

Glukhov

Cattaneo

et al. 2023

IEEE J. Electron Devices Soc.

View full text Add to dashboard Cite

In-memory computing (IMC) aims at executing numerical operations via physical processes, such as current summation and charge collection, thus accelerating common computing tasks including the matrix-vector multiplication. While extremely promising for memory-intensive processing such as machine learning and deep learning, the IMC design and realization must face significant challenges due to device and circuit nonidealities. This work provides an overview of the research trends and options for IMC-based implementations of deep learning accelerators with emerging memory technologies. The device technologies, the computing primitives, and the digital/analog/mixed design approaches are presented. Finally, the major device issues and metrics for IMC are discussed and benchmarked.

show abstract

“…For example, the block-wise weight sparsity and dynamic activation sparsity is proposed to apply sparsity techniques on the regular CIM structure [14] . The set-associate block-wise sparsity, tensor-train compression and bitwise sparsity are also explored to save execution time or power consumption [15,26] .…”

Section: System-level Cim Chipsmentioning

confidence: 99%

Trending IC design directions in 2022

Chan¹,

Cheng²,

Deng³

et al. 2022

J. Semicond.

View full text Add to dashboard Cite

For the non-stop demands for a better and smarter society, the number of electronic devices keeps increasing exponentially; and the computation power, communication data rate, smart sensing capability and intelligence are always not enough. Hardware supports software, while the integrated circuit (IC) is the core of hardware. In this long review paper, we summarize and discuss recent trending IC design directions and challenges, and try to give the readers big/cool pictures on each selected small/hot topics. We divide the trends into the following six categories, namely, 1) machine learning and artificial intelligence (AI) chips, 2) communication ICs, 3) data converters, 4) power converters, 5) imagers and range sensors, 6) emerging directions. Hope you find this paper useful for your future research and works.

show abstract

A 28nm 29.2TFLOPS/W BF16 and 36.5TOPS/W INT8 Reconfigurable Digital CIM Processor with Unified FP/INT Pipeline and Bitwise In-Memory Booth Multiplication for Cloud Deep Learning Acceleration

Cited by 63 publications

References 2 publications

Hadamard product-based in-memory computing design for floating point neural network training

Hadamard product-based in-memory computing design for floating point neural network training

In-Memory Computing for Machine Learning and Deep Learning

Trending IC design directions in 2022

Contact Info

Product

Resources

About