Bit-pragmatic deep neural network computing

Abstract

Abstract-We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an architecture that exploits it improving performance and energy efficiency. The source of these ineffectual computations is best understood in the context of conventional multipliers which generate internally multiple terms, that is, products of the multiplicand and powers of two, which added together produce the final product [1]. At runtime, many of these terms are zero as they are generated when the multiplicand is combined with the zero-bits of the multiplicator. While conventional bit-parallel multipliers calculate all terms in parallel to reduce individual product latency, PRA calculates only the non-zero terms using a) on-thefly conversion of the multiplicator representation into an explicit list of powers of two, and b) bit-parallel multplicand/bit-serial multiplicator processing units.PRA exploits two sources of ineffectual computations: 1) the aforementioned zero product terms which are the result of the lack of explicitness in the multiplicator representation, and 2) the excess in the representation precision used for both multiplicants and multiplicators, e.g., [2]. Measurements demonstrate that for the convolutional layers, a straightforward variant of PRA improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by 1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28% and 10% on average compared to DaDN and STR. An improved cross lane synchronization scheme boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits persist even with an 8-bit quantized representation [5].

Architecting Effectual Computation for Machine Learning Accelerators

“…We use two baselines: the 1 st one is DaDianNao [9], the de facto design to report relative performance of diverse DCNN accelerators; the 2 nd one is a state-of-the-art bit-serial implementation [5] (PRA), it is also designed for computing essential bits of the activations, and we enroll its fp16 design on weights for fair comparison. We implement Tetris with two configurable modes, fp16 and int8, as mentioned in Section III.…”

Adaptive Loss-Aware Quantization for Multi-Bit Networks

“…This will significantly speed up the convolution operations [16]. (ii) A network quantized to fixed-point requires specialized integer arithmetic units (with various bitwidth) for efficient computing [1,18], whereas a network quantized with multiple binary bases adopts the same operations mentioned before as binary networks. Popular networks quantized with binary bases include Binary Networks and Multi-bit Networks.…”

Analyzing Machine Learning Workloads Using a Detailed GPU Simulator

“…2) Simulation: Some prior work has also simulated machine learning workloads, but these papers used private simulators [33]- [36]. Since these simulators are not publicly available and few details are available, it is difficult to compare their approaches to ours.…”