Verified instruction-level energy consumption measurement for NVIDIA GPUs

Arafa, Yehia; ElWazir, Ammar; Elkanishy, Abdelrahman; Aly, Youssef; Elsayed, Ayatelrahman; Badawy, Abdel-Hameed A.; Chennupati, Gopinath; Eidenbenz, Stephan; Santhi, Nandakishore

doi:10.1145/3387902.3392613

Cited by 24 publications

(14 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accuracy of the built-in sensors was investigated by Farad et al [27] who found that the average mean error using an abstract model of a GPU is about 10% compared to measurements using external power meters. This error value was confirmed by Arafa et al [28] who measured the energy consumption of almost all PTX instructions for four generations of NVIDIA GPUs. They have found that the Maxwell and the Turing generations of GPUs have high energy consumption when compared to the Pascal and the Volta generations of NVIDIA GPUs which are found to be more energy efficient.…”

Section: Related Worksupporting

confidence: 53%

Efficiency Near the Edge: Increasing the Energy Efficiency of FFTs on GPUs for Real-Time Edge Computing

Adámek¹,

Novotný²,

Thiyagalingam³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

The Square Kilometre Array (SKA) is an international initiative for developing the world's largest radio telescope with a total collecting area of over a million square meters. The scale of the operation, combined with the remote location of the telescope, requires the use of energy-efficient computational algorithms. This, along with the extreme data rates that will be produced by the SKA and the requirement for a real-time observing capability, necessitates in-situ data processing in an edge style computing solution. More generally, energy efficiency in the modern computing landscape is becoming of paramount concern. Whether it be the power budget that can limit some of the world's largest supercomputers, or the limited power available to the smallest Internet-of-Things devices. In this paper, we study the impact of hardware frequency scaling on the energy consumption and execution time of the Fast Fourier Transform (FFT) on NVIDIA GPUs using the cuFFT library. The FFT is used in many areas of science and it is one of the key algorithms used in radio astronomy data processing pipelines. Through the use of frequency scaling, we show that we can lower the power consumption of the NVIDIA A100 GPU when computing the FFT by up to 47% compared to the boost clock frequency, with less than a 10% increase in the execution time. Furthermore, using one common core clock frequency for all tested FFT lengths, we show on average a 43% reduction in power consumption compared to the boost core clock frequency with an increase in the execution time still below 10%. We demonstrate how these results can be used to lower the power consumption of existing data processing pipelines. These savings, when considered over years of operation, can yield significant financial savings, but can also lead to a significant reduction of greenhouse gas emissions.

show abstract

Section: Related Worksupporting

confidence: 53%

Efficiency Near the Edge: Increasing the Energy Efficiency of FFTs on GPUs for Real-Time Edge Computing

Adámek¹,

Novotný²,

Thiyagalingam³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…After that, we try frequencies around the predicted optimal configuration and compare the measured energy-time data to search for the actual optimal configuration ( 7 ). Finally, we set the actual optimal configuration and monitor energy characteristics ( 8 ).…”

Section: Framework Overviewmentioning

confidence: 99%

“…The more similar the curves are, the closer the candidate period is to the actual period. So we evaluate the similarity of each pair of adjacent sub-curves (line [5][6][7][8][9][10][11][12][13][14][15][16][17][18].…”

Section: Feature Sequence Similarity: Find Accurate Periodsmentioning

confidence: 99%

“…The CUPTI [15] supports profiling over 1100 performance counters on NVIDIA high-end GPUs, such as RTX2080Ti and RTX3080Ti. Inspired by Arafa's work [7], we treat each Parallel Thread Execution (PTX) instruction as a basic unit of energy consumption and focus on performance counter metrics that reflect the density of different types of PTX instructions. We list the selected metrics in Table 2.…”

Section: Feature Selectionmentioning

confidence: 99%

“…Most of the existing studies in general GPU energy optimization are offline techniques [3]- [7]. These techniques require profiling applications ahead of time [3]- [5], [7] or analyzing their source code [6] in advance to collect application features. These approaches cannot adapt to the change of the program behavior and can incur expensive profiling overhead.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic GPU Energy Optimization for Machine Learning Training Workloads

Wang

Zhang

Lai

et al. 2022

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

GPUs are widely used to accelerate the training of machine learning workloads. As modern machine learning models become increasingly larger, they require a longer time to train, leading to higher GPU energy consumption. This paper presents GPOEO, an online GPU energy optimization framework for machine learning training workloads. GPOEO dynamically determines the optimal energy configuration by employing novel techniques for online measurement, multi-objective prediction modeling, and search optimization. To characterize the target workload behavior, GPOEO utilizes GPU performance counters. To reduce the performance counter profiling overhead, it uses an analytical model to detect the training iteration change and only collects performance counter data when an iteration shift is detected. GPOEO employs multi-objective models based on gradient boosting and a local search algorithm to find a trade-off between execution time and energy consumption. We evaluate the GPOEO by applying it to 71 machine learning workloads from two AI benchmark suites running on an NVIDIA RTX3080Ti GPU. Compared with the NVIDIA default scheduling strategy, GPOEO delivers a mean energy saving of 16.2% with a modest average execution time increase of 5.1%.

show abstract

Power overwhelming: the one with the oscilloscopes

Gralka,

Müller,

Heinemann

et al. 2024

J Vis

View full text Add to dashboard Cite

Visualization as a discipline has to investigate its practical implications in a world steadily moving toward greener computing methods. Quantifying the power consumption of visualization algorithms is thus essential, given the ever-increasing energy needs of GPUs. Previous approaches rely on integrated sensors or invasive methods that require modifications and special test setups. However, they still suffer from imprecision from low sampling rates and integration over time. Using a high-precision, high-frequency setup via steerable oscilloscopes, we can objectively measure the resulting quality of previous approaches. This is essential to establish a ground truth, pave the way for improved modeling of power consumption in general, and enable better estimates based on the output of lower-quality sensors. We finally discuss benefits that can be drawn from the additional insight of the higher-precision setup and which additional use cases can justify the incurred costs. Graphical abstract

show abstract

Verified instruction-level energy consumption measurement for NVIDIA GPUs

Cited by 24 publications

References 30 publications

Efficiency Near the Edge: Increasing the Energy Efficiency of FFTs on GPUs for Real-Time Edge Computing

Efficiency Near the Edge: Increasing the Energy Efficiency of FFTs on GPUs for Real-Time Edge Computing

Dynamic GPU Energy Optimization for Machine Learning Training Workloads

Power overwhelming: the one with the oscilloscopes

Contact Info

Product

Resources

About