Throughput-Distortion Computation of Generic Matrix Multiplication: Toward a Computation Channel for Digital Signal Processing Systems

Abstract

The generic matrix multiply (GEMM) function is the core element of high-performance linear algebra libraries used in many computationally-demanding digital signal processing (DSP) systems. We propose an acceleration technique for GEMM based on dynamically adjusting the imprecision (distortion) of computation. Our technique employs adaptive scalar companding and rounding to input matrix blocks followed by two forms of packing in floating-point that allow for concurrent calculation of multiple results. Since the adaptive companding process controls the increase of concurrency (via packing), the increase in processing throughput (and the corresponding increase in distortion) depends on the input data statistics. To demonstrate this, we derive the optimal throughputdistortion control framework for GEMM for the broad class of zero-mean, independent identically distributed, input sources. Our approach converts matrix multiplication in programmable processors into a computation channel: when increasing the processing throughput, the output noise (error) increases due to (i) coarser quantization and (ii) computational errors caused by exceeding the machine-precision limitations. We show that, under certain distortion in the GEMM computation, the proposed framework can significantly surpass 100% of the peak performance of a given processor. The practical benefits of our proposal are shown in a face recognition system and a multi-layer perceptron system trained for metadata learning from a large music feature database.

Error Tolerant Multimedia Stream Processing: There's Plenty of Room at the Top (of the System Stack)

“…The aim is to provide for graceful resilience and performance scaling according to transmission rates and the tolerated signal distortion. Following this analogy, perhaps it is time to consider computational multimedia stream processing as a computation channel [6], [7] or as a stochastic computing system [8], [9] optimizing for expected performance and not for the worst case. Instead of solely striving for advances in "channel quality" (i.e., increasing fault-free CMOS integration and processor operating frequencies at substantial cost and complexity), we can instead design multimedia processing in software and hardware that can withstand certain error rates at all layers of the system stack [1], [2], [4].…”

“…Similar to communications systems, the aim would be to obtain graceful and resilient approximations of the output results with increased processing (i.e., "channel") resources. Owing to their potential error-tolerance capability, many multimedia stream processing applications currently have enormous scaling potential left unexploited [1], [6]- [8], [10], [11].…”

“…For example, game-theoretic optimization of distributed classifier chains under precision-complexity constraints was shown to achieve significant resource-precision scalability for speaker recognition from voice recordings [63]. Asanovic et al [64] and Anastasia and Andreopoulos [6] demonstrated that back-propagation learning algorithms with matrix operations are robust to significant noise levels in the performed computations. Specifically, it has been found that back-propagation learning can be implemented reliably, even when the update process is converted from floating-point to 16-bit fixed-point representation [64], or to noisy computation by companding and packing with (approximately) 24 dB signal-to-noise ratio (against the single-precision floating-point results [6]).…”

“…Asanovic et al [64] and Anastasia and Andreopoulos [6] demonstrated that back-propagation learning algorithms with matrix operations are robust to significant noise levels in the performed computations. Specifically, it has been found that back-propagation learning can be implemented reliably, even when the update process is converted from floating-point to 16-bit fixed-point representation [64], or to noisy computation by companding and packing with (approximately) 24 dB signal-to-noise ratio (against the single-precision floating-point results [6]). Since the latter approach is an emerging approach proposed recently, it is reviewed in more detail in the next section.…”

FPGA Implementation of Latency, Computational time Improvements in Matrix Multiplication

“…The increases in the density and speed of field-programmable gate arrays (FPGAs) [1] make them attractive as flexible and high-speed alternatives to DSPs [3] and ASICs. It is a highly procedure oriented computation [6], there is only one way to multiply two matrices and it involves lots of multiplications and additions.…”