A Framework for Automated and Controlled Floating-Point Accuracy Reduction in Graphics Applications on GPUs

Abstract

Reducing the precision of floating-point values can improve performance and/or reduce energy expenditure in computer graphics, among other, applications. However, reducing the precision level of floating-point values in a controlled fashion needs support both at the compiler and at the microarchitecture level. At the compiler level, a method is needed to automate the reduction of precision of each floating-point value. At the microarchitecture level, a lower precision of each floating-point register can allow more floating-point values to be packed into a register file. This, however, calls for new register file organizations.This article proposes an automated precision-selection method and a novel GPU register file organization that can store floating-point register values at arbitrary precisions densely. The automated precision-selection method uses a data-driven approach for setting the precision level of floating-point values, given a quality threshold and a representative set of input data. By allowing a small, but acceptable, degradation in output quality, our method can remove a significant amount of the bits needed to represent floating-point values in the investigated kernels (between 28% and 60%). Our proposed register file organization exploits these lowerprecision floating-point values by packing several of them into the same physical register. This reduces the register pressure per thread by up to 48%, and by 27% on average, for a negligible output-quality degradation. This can enable GPUs to keep up to twice as many threads in flight simultaneously.A. Angerd et al.Narrowing the width of floating-point values is an effective approach to both achieve higher performance [6] as well as higher energy efficiency [8,16,22], especially for GPUs, which now are supporting 16-bit floating-point standards [14]. A substantially narrower width of floatingpoint values can open up many novel optimization approaches at the hardware level, such as more resource-efficient register files, data-paths, functional units, and cache memory subsystems. However, to leverage such optimizations, two issues must be addressed. First, the width of each and every floating-point value must be established at the instruction level. Second, architectural support is needed to use the established widths to utilize register file, data path, functional unit, or cache resources more efficiently. The goal of this article is to provide such a framework.Programming language models that enable approximate computing, such as EnerJ [20] and Flex-Java [15], take a binary view to declare a variable as either approximable or precise. Hence, they cannot deal with an arbitrary width of floating-point variables. Even if there were support for specifying precision, it would be laborious or nearly impossible for programmers to use it efficiently. Also, it would need support at the instruction-set architecture level, such as in Quora [24], to specify error-bounds at the instruction level.Precimonious [18] provides a framework to automatically select among...

Avr

“…Reducing the precision of floating point [6,21,42] and fixed point [22] numbers has been used to alleviate the memory bandwidth bottleneck in deep neural networks [22], GPU workloads [42] and other approximation tolerant applications [21], thereby improving performance and energy efficiency. However, the compression ratio is still limited between 2:1 and 4:1 despite the loss of precision as these approaches do not exploit inter-value similarities to compress data.…”

“…Approximate deduplication of individual cachelines increases cache capacity [39], however, multiple values need to match at cacheline granularity. A form of lossy compression has been applied in approximate computing, but is constrained to reducing precision of single values truncating their least significant bits [6,21,22,42] and therefore achieves limited compression ratio.…”

Energy-Efficient Iterative Refinement Using Dynamic Precision

“…Automated precision tuning for an application was investigated in [3], [4]. The algorithm of [3] adapts the delta debugging based search algorithm to seek 1-minimal test case (e.g., for 1-minimal test case, replacing any variable with a lower precision violates either accuracy constraint or performance constraint).…”

“…The algorithm of [3] adapts the delta debugging based search algorithm to seek 1-minimal test case (e.g., for 1-minimal test case, replacing any variable with a lower precision violates either accuracy constraint or performance constraint). Another automated precision tuning research was proposed in [4] to investigate precision tuning for a lower level implementation.…”

“…Recent work has explored automated precision tuning tools [3], [4]. Unfortunately, these tools have not proven practical but several reasons.…”