Predictable GPUs Frequency Scaling for Energy and Performance

Abstract

Dynamic voltage and frequency scaling (DVFS) is an important solution to balance performance and energy consumption, and hardware vendors provide management libraries that allow the programmer to change both memory and core frequencies. The possibility to manually set these frequencies is a great opportunity for application tuning, which can focus on the best applicationdependent setting. However, this task is not straightforward because of the large set of possible configurations and because of the multi-objective nature of the problem, which minimizes energy consumption and maximizes performance. This paper proposes a method to predict the best core and memory frequency configurations on GPUs for an input OpenCL kernel. Our modeling approach, based on machine learning, first predicts speedup and normalized energy over the default frequency configuration. Then, it combines the two models into a multi-objective one that predicts a Pareto-set of frequency configurations. The approach uses static code features, is built on a set of carefully designed microbenchmarks, and can predict the best frequency settings of a new kernel without executing it. Test results show that our modeling approach is very accurate on predicting extrema points and Pareto set for ten out of twelve test benchmarks, and discover frequency configurations that dominate the default configuration in either energy or performance. CCS CONCEPTS • Computer systems organization → Parallel architectures; • Hardware → Power and energy.

GPU Static Modeling Using PTX and Deep Structured Learning

“…al. [12], is to consider a multiobjective optimization problem, with a set of Pareto-optimal solutions. In other words, one could search for the V-F configurations that maximize the speedup and minimize the normalized energy, i.e., the configurations that are not dominated by any other configuration.…”

“…In particular, Alavani et al [10] presented a way to predict the execution time of an application prior to its execution, with an average prediction error of 26.9% on a Tesla K20 GPU (Kepler). On the other hand, Fan et al [12] developed DVFS-aware static models for performance and energy of GPU devices. The two models are trained based on a static vector of 10 features, where each component represents the count of a type of instructions.…”

“…The main use-case of predictive models, such as the ones herein proposed, is to perform the DVFS management to maximize the energy efficiency of the computing system. Considering a multi-objective optimization problem with a set of Pareto-optimal solutions, similar to the one that was proposed in [12], this technique can be a useful approach to find the best V-F configurations for different applications. Fig.…”

“…An alternative and highly promising approach consists in providing predictions of the DVFS impact on the application behavior, prior to its execution. This alternative relies on using the GPU assembly of the kernels 2 [10]- [12] (described in the NVIDIA PTX ISA [13]), which can be obtained at compile-time. Although this approach is expected to yield less accurate results (when compared with state-of-the-art run-time models), it allows the first execution of an application to be done at a close to the optimal V-F configuration.…”

“…To that end, the proposed methodology uses the PTX assembly code given by the compiler. However, unlike previous works that simply rely on general code statistics, such as the histogram of instructions in the PTX code [12], [14], the proposed approach takes a step further and considers the specific sequence of kernel instructions, to improve the prediction accuracy. To model how the pattern of instructions stresses the GPU components, thus contributing to different performance, power and energy scalings, a deep neural network is used.…”