Proteus

Abstract

This work exploits the tolerance of Deep Neural Networks (DNNs) to reduced precision numerical representations and specifically, their recently demonstrated ability to tolerate representations of di↵erent precision per layer while maintaining accuracy. This flexibility enables improvements over conventional DNN implementations that use a single, uniform representation. This work proposes Proteus, which reduces the data tra c and storage footprint needed by DNNs, resulting in reduced energy and improved area efficiency for DNN implementations. Proteus uses a di↵er-ent representation per layer for both the data (neurons) and the weights (synapses) processed by DNNs. Proteus is a layered extension over existing DNN implementations that converts between the numerical representation used by the DNN execution engines and the shorter, layer-specific fixedpoint representation used when reading and writing data values to memory be it on-chip bu↵ers or o↵-chip memory.Proteus uses a novel memory layout for DNN data, enabling a simple, low-cost and low-energy conversion unit.We evaluate Proteus as an extension to a state-of-the-art accelerator [7] which uses a uniform 16-bit fixed-point representation. On five popular DNNs Proteus reduces data tra c among layers by 43% on average while maintaining accuracy within 1% even when compared to a single precision floating-point implementation. As a result, Proteus improves energy by 15% with no performance loss. Proteus also reduces the data footprint by at least 38% and hence the amount of on-chip bu↵ering needed resulting in an implementation that requires 20% less area overall. This area savings can be used to improve cost by building smaller chips, to process larger DNNs for the same on-chip area, or to incorporate an additional three execution engines increasing peak performance bandwidth by 18%.

Dynamic Multi-Resolution Data Storage

“…Even though VS shares the benefits from recent advances in ISP [6,9,14,23,31,33,35,40,67,75,76,82,85,89,96] and neardata processing [2,17,20,39,48,50,63,80,84], these frameworks need the mechanisms that VS offers in order to execute approximate computing applications efficiently. And while using approximate computing in channel encoding [38,62] and memory controller [30] can achieve an effect similar to that of VS in terms of reducing data-movement overhead, VS is independent of these projects and requires no changes in hardware.…”

Maximizing CNN Accelerator Efficiency Through Resource Partitioning

“…[1] proposes a CNN accelerator design that can skip computations on input values that are zeros. [14,21] reduce an accelerator's bandwidth and buffer use. [21] uses per-layer data quantization and matrix-decomposition, whereas [14] uses perlayer numerical precision reduction.…”

“…[14,21] reduce an accelerator's bandwidth and buffer use. [21] uses per-layer data quantization and matrix-decomposition, whereas [14] uses perlayer numerical precision reduction. [2] uses a fused-layer technique to reduce bandwidth use of convolutional layers.…”