What does Power Consumption Behavior of HPC Jobs Reveal? : Demystifying, Quantifying, and Predicting Power Consumption Characteristics

Patel, Tirthak; Wagenhäuser, Adam; Eibel, Christopher; Hönig, Timo; Zeiser, Thomas; Tiwari, Devesh

doi:10.1109/ipdps47924.2020.00087

Cited by 25 publications

(5 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Patel et al [5] explore the power characteristics of typical HPC jobs during the approach to the Exascale era. As HPC systems become increasingly power constrained, they show that a data-driven approach to HPC application power characteristics can be used to make more effective use of HPC systems.…”

Section: Related Workmentioning

confidence: 99%

Understanding Power and Energy Utilization in Large Scale Production Physics Simulation Codes

Ryujin¹,

Vargas²,

Karlin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Power is an often-cited reason for moving to advanced architectures on the path to Exascale computing. This is due to the practical concern of delivering enough power to successfully site and operate these machines, as well as concerns over energy usage while running large simulations. Since accurate power measurements can be difficult to obtain, processor thermal design power (TDP) is a possible surrogate due to its simplicity and availability. However, TDP is not indicative of typical power usage while running simulations. Using commodity and advance technology systems at Lawrence Livermore National Laboratory (LLNL) and Sandia National Laboratory, we performed a series of experiments to measure power and energy usage in running simulation codes. These experiments indicate that large scale LLNL simulation codes are significantly more efficient than a simple processor TDP model might suggest.

show abstract

Section: Related Workmentioning

confidence: 99%

Understanding Power and Energy Utilization in Large Scale Production Physics Simulation Codes

Ryujin¹,

Vargas²,

Karlin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, datacenters can expose thousands of signals, so even our broad selection imposes a bias. Finding a complete and general, holistic method of analysis is beyond the scope of this work-a goal which we envision for the entire community, for the next decade, which already includes awardwinning work that focuses on selecting meaningful signals [69] and large-scale data collection [62,44,40]. Furthermore, the method proposed here can be contrasted with methods from the other end of the holistic-reductionist spectrum; compared with focused work on even one of the questions we address, our method cannot produce the same depth for the same effort.…”

Section: Known Limitationsmentioning

confidence: 99%

“…From the relatively few traces that are shared publicly, many are focused on important but specific kinds of workloads, such as tightly-coupled parallel jobs [18], bags of tasks [26], and workflows [65]. Other datasets only include a limited subset of metrics such as power consumption [44], or high-level job information [43]. Only a handful of datasets include low-level server metrics, such as the Microsoft Azure serverless traces [49] or the Solvinity business-critical traces [50].…”

Section: Introductionmentioning

confidence: 99%

A Holistic Analysis of Datacenter Operations: Resource Usage, Energy, and Workload Characterization -- Extended Technical Report

Versluis,

Cetin,

Greeven

et al. 2021

Preprint

View full text Add to dashboard Cite

Improving datacenter operations is vital for the digital society. We posit that doing so requires our community to shift, from operational aspects taken in isolation to holistic analysis of datacenter resources, energy, and workloads. In turn, this shift will require new analysis methods, and open-access, FAIR datasets with fine temporal and spatial granularity. We leverage in this work one of the (rare) public datasets providing fine-grained information on datacenter operations. Using it, we show strong evidence that fine-grained information reveals new operational aspects. We then propose a method for holistic analysis of datacenter operations, providing statistical characterization of node, energy, and workload aspects. We demonstrate the benefits of our holistic analysis method by applying it to the operations of a datacenter infrastructure with over 300 nodes. Our analysis reveals both generic and ML-specific aspects, and further details how the operational behavior of the datacenter changed during the 2020 COVID-19 pandemic. We make over 30 main observations, providing holistic insight into the long-term operation of a large-scale, public scientific infrastructure. We suggest such observations can help immediately with performance engineering tasks such as predicting future datacenter load, and also long-term with the design of datacenter infrastructure.

show abstract

“…As deep learning (DL) workflows become more prevalent in the sciences, neural architecture search and hyperparameter sweeps consume an increasingly enormous * ncfrey@mit.edu amount of compute and power resources [3,5,27,28,39,44,48] at high-performance computing (HPC) centers [2,41,46] and cloud providers. While the cost per training step has decreased for deep neural networks (DNNs) due to optimized hardware and backend optimizations, overall costs have increased and training large models can reach into the millions of dollars [39].…”

Section: Introductionmentioning

confidence: 99%

Benchmarking Resource Usage for Efficient Distributed Deep Learning

Frey¹,

Li²,

McDonald³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning (DL) workflows demand an everincreasing budget of compute and energy in order to achieve outsized gains. Neural architecture searches, hyperparameter sweeps, and rapid prototyping consume immense resources that can prevent resource-constrained researchers from experimenting with large models and carry considerable environmental impact. As such, it becomes essential to understand how different deep neural networks (DNNs) and training leverage increasing compute and energy resources-especially specialized computationally-intensive models across different domains and applications.In this paper, we conduct over 3,400 experiments training an array of deep networks representing various domains/tasks-natural language processing, computer vision, and chemistry-on up to 424 graphics processing units (GPUs). During training, our experiments systematically vary compute resource characteristics and energy-saving mechanisms such as power utilization and GPU clock rate limits to capture and illustrate the different trade-offs and scaling behaviors each representative model exhibits under various resource and energyconstrained regimes. We fit power law models that describe how training time scales with available compute resources and energy constraints. We anticipate that these findings will help inform and guide highperformance computing providers in optimizing resource utilization, by selectively reducing energy consumption for different deep learning tasks/workflows with minimal impact on training.

show abstract

What does Power Consumption Behavior of HPC Jobs Reveal? : Demystifying, Quantifying, and Predicting Power Consumption Characteristics

Cited by 25 publications

References 55 publications

Understanding Power and Energy Utilization in Large Scale Production Physics Simulation Codes

Understanding Power and Energy Utilization in Large Scale Production Physics Simulation Codes

A Holistic Analysis of Datacenter Operations: Resource Usage, Energy, and Workload Characterization -- Extended Technical Report

Benchmarking Resource Usage for Efficient Distributed Deep Learning

Contact Info

Product

Resources

About