FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support

Abstract

Training deep neural networks (DNNs) is a computationally expensive job, which can take weeks or months even with high performance GPUs. As a remedy for this challenge, community has started exploring the use of more efficient data representations in the training process, e.g., block floating point (BFP). However, prior work on BFP-based DNN accelerators rely on a specific BFP representation making them less versatile. This paper builds upon an algorithmic observation that we can accelerate the training by leveraging multiple BFP precisions without compromising the finally achieved accuracy. Backed up by this algorithmic opportunity, we develop a flexible DNN training accelerator, dubbed FlexBlock, which supports three different BFP precision modes, possibly different among activation, weight, and gradient tensors. While several prior works proposed such multi-precision support for DNN accelerators, not only do they focus only on the inference, but also their core utilization is suboptimal at a fixed precision and specific layer types when the training is considered. Instead, FlexBlock is designed in such a way that high core utilization is achievable for i) various layer types, and ii) three BFP precisions by mapping data in a hierarchical manner to its compute units. We evaluate the effectiveness of FlexBlock architecture using well-known DNNs on CIFAR, ImageNet and WMT14 datasets. As a result, training in FlexBlock significantly improves the training speed by 1.5∼5.3× and the energy efficiency by 2.4∼7.0× on average compared to other training accelerators and incurs marginal accuracy loss compared to full-precision training.

LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training

“…The prior work on designing an energy-efficient DNN accelerator mostly focus on Conv/FC operations [26], [27], [34], [36], while there is lack of research on making the BN hardware more efficient. One of the most effective ways of improving hardware efficiency of a processing unit is reducing the bit-precision.…”

“…With the mixed-precision training, multiplications are performed in FP16 while accumulations are performed in FP32. Unconventional data representations suited at DNN training, such as bfloat16 [13], [19] and block floating point (BFP) representation [8], [26], have been studied as well. To enable DNN training at much lower hardware cost, possibly at the edge, researchers have explored low-precision training using FP8 (i.e., 8-bit floating point) with the support of squeeze and shift operations [4] or exponent biases [9] to cover a wide dynamic range of the original data distribution.…”