Understanding GPU Power

Abstract

Modern graphics processing units (GPUs) have complex architectures that admit exceptional performance and energy efficiency for high-throughput applications. Although GPUs consume large amounts of power, their use for high-throughput applications facilitate state-of-the-art energy efficiency and performance. Consequently, continued development relies on understanding their power consumption. This work is a survey of GPU power modeling and profiling methods with increased detail on noteworthy efforts. As direct measurement of GPU power is necessary for model evaluation and parameter initiation, internal and external power sensors are discussed. Hardware counters, which are low-level tallies of hardware events, share strong correlation to power use and performance. Statistical correlation between power and performance counters has yielded worthwhile GPU power models, yet the complexity inherent to GPU architectures presents new hurdles for power modeling. Developments and challenges of counter-based GPU power modeling are discussed. Often building on the counter-based models, research efforts for GPU power simulation, which make power predictions from input code and hardware knowledge, provide opportunities for optimization in programming or architectural design. Noteworthy strides in power simulations for GPUs are included along with their performance or functional simulator counterparts when appropriate. Last, possible directions for future research are discussed.

Accurate Calibration of Power Measurements from Internal Power Sensors on NVIDIA Jetson Devices

“…• Hardware setup: As indicated in [8], the sampling rate of the power measurements directly impacts their precision for GPU-enabled edge computers. The hardware used in this study consists of a set of commercially available NVIDIA Jetson devices and a high sampling rate external oscilloscope to accurately measure, store and stream power consumption data; the measurement setup is depicted in Fig.…”

“…However, built-in sensors typically measure average values, while failing to track current peaks. Furthermore, the sampling rate impacts the precision of the measured data [8], and concurrently running programs impacts the Central Processing unit (CPU) access frequency to read from the onboard sensors, which, in turn, impacts the accuracy of the measured power. From our measurements, we do not think that the reason for this gap is attributable to the sampling frequency, as it is observed even when the power is nearly constant.…”

Proceedings of the 52nd International Conference on Parallel Processing

“…For instance, GPU power modeling can benefit from HighRPM, as it shares similarities with CPU power modeling. However, GPU power modeling often requires additional considerations specific to the architecture [15]. Adapting HighRPM to GPU would involve adjusting the model to utilize GPU performance counters and collecting training data on the target platform.…”

Design of a Standard and Programmatically Accessible Interface for Smart Meters to Allow Monitoring Automation of the Energy Consumed by the Execution of Computer Software

“…Therefore, estimation models are commonly used for large computer systems, such as datacenters [18,19], rather than personal computers, because measuring their actual overall consumption is more complicated than estimating it with a reasonable error rate. Surveys on energy estimation models have also been reported in other fields, such as on GPUs [20], multicore processors [21], mobile devices [22,23], HPC systems [24], or for the execution of machine learning algorithms [16,17,25].…”