BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computing

Abstract

Matrix-matrix multiplication is a key computational kernel for numerous applications in science and engineering, with ample parallelism and data locality that lends itself well to high-performance implementations. Many matrix multiplicationdependent applications can use reduced-precision integer or fixedpoint representations to increase their performance and energy efficiency while still offering adequate quality of results. However, precision requirements may vary between different application phases or depend on input data, rendering constant-precision solutions ineffective. We present BISMO, a vectorized bitserial matrix multiplication overlay for reconfigurable computing. BISMO utilizes the excellent binary-operation performance of FPGAs to offer a matrix multiplication performance that scales with required precision and parallelism. We characterize the resource usage and performance of BISMO across a range of parameters to build a hardware cost model, and demonstrate a peak performance of 6.5 TOPS on the Xilinx PYNQ-Z1 board.

Improving Memory Access Locality for Vectorized Bit-Serial Matrix Multiplication in Reconfigurable Computing

“…Algorithm 1 represents a straight forward algorithm by which bit-serial matrix multiplication can be performed [6], with Equation 1 representing how a single element of the result matrix is calculated. Priority is given to the calculation of the inner product of two binary matrices (e.g., L [0] • R [0] ) before proceeding to calculate the inner product of another bit-precision (e.g., L [0] • R [1] ).…”

“…An accelerator optimized for performing binary matrix multiplication is not sufficient to take advantage of the locality-aware scheduling algorithm described in the previous section (Algorithm 2). BISMO [6] was designed for efficient computation of binary matrix multiplications but due to its software programmability it provides the necessary flexibility to evaluate a large variety of different scheduling algorithms.…”

“…For detailed information on BISMO and its software programmability please refer to the article by Y. Umuroglu et al [6].…”

“…However, bitserial computations can be highly beneficial for low-precision matrix multiplication as the latency is low due to the lowprecision and is offset by the inherent data-level parallelism in matrix multiplications. Parallelism can be obtained not only through instantiation of multiple bit-serial units but also through vectorization as demonstrated in BISMO [6].…”

Hardware-Centric AutoML for Mixed-Precision Quantization

“…This flexibility was originally not supported by chip vendors until recently the hardware manufacturers started to implement this feature: Apple released the A12 Bionic chip that supports flexible bits for the neural network inference (Apple 2018); NVIDIA recently introduced the Turing GPU architecture that supports 1-bit, 4-bit, 8-bit and 16-bit arithmetic operations (Nvidia 2018); Imagination launched a flexible neural network IP that supports per-layer bitwidth adjustment for both weights and activations (Imagination 2018). Besides industry, recently academia also works on the bit-level flexible hardware design: BISMO (Umuroglu et al 2018) proposed the bit-serial multiplier to support multiplications of 1 to 8 bits; BitFusion (Sharma et al 2018) supports multiplications of 2, 4, 8 and 16 bits in a spatial manner.…”