Online Power Management for Multi-Cores: A Reinforcement Learning Based Approach

Abstract

Power and energy is the first-class design constraint for multi-core processors and is a limiting factor for future-generation supercomputers. While modern processor design provides a wide range of mechanisms for power and energy optimization, it remains unclear how software can make the best use of them. This paper presents a novel approach for runtime power optimization on modern multi-core systems. Our policy combines power capping and uncore frequency scaling to match the hardware power profile to the dynamically changing program behavior at runtime. We achieve this by employing reinforcement learning (RL) to automatically explore the energy-performance optimization space from training programs, learning the subtle relationships between the hardware power profile, the program characteristics, power consumption and program running times. Our RL framework then uses the learned knowledge to adapt the chip's power budget and uncore frequency to match the changing program phases for any new, previously unseen program. We evaluate our approach on two computing clusters by applying our techniques to 11 parallel programs that were not seen by our RL framework at the training stage. Experimental results show that our approach can reduce the system-level energy consumption by 12%, on average, with less than 3% of slowdown on the application performance. By lowering the uncore frequency to leave more energy budget to allow the processor cores to run at a higher frequency, our approach can reduce the energy consumption by up to 17% while improving the application performance by 5% for specific workloads.

Real-time task scheduling and network device security for complex embedded systems based on deep learning networks

“…Recent research efforts have focused on how to improve this architecture. Methods include: enabling more performance counters to build complex system features [19] [20] [27], using powerful models for prediction [19] [18] [20] [21] [24], designing better control rules [19] [18] [20] [21], or learning control policy based on reinforcement learning [24] [25] [26] [27] [30].…”

“…Based on the previous work, Ramegowda et al [42] implemented and validated the hybrid DVFS method in various embedded devices running the Linux system. Wang et al [27] used Double Q learning to explore the energy-performance optimization for both CPU core and uncore parts. Specifically, they used the instruction per cycle (IPC), and the misses per operation (MPO) [43] as the state measurement of the environment and used IP C 3 W as the reward to describe the tradeoff between energy and performance.…”

Combining Uncore Frequency and Dynamic Power Capping to Improve Power Savings

“…RELATED WORK There are many studies which rely on power capping, however very few studies rely on dynamic power capping while fewer studies combine uncore frequency scaling to dynamic power capping. In [32] the authors propose to rely on reinforcement learning to get the best energy consumption with uncore frequency and power capping. Instruction Per Cycles (IPC) are used to control performance loss.…”

DEPO: A dynamic energy‐performance optimizer tool for automatic power capping for energy efficient high‐performance computing

“…READEX (Runtime Exploitation of Application Dynamism for Energy-efficient eXascale computing) OpenMP and MPI code instrumenting tools for optimization of energy-aware HPC computing 29 A multi-agent based intelligent energy management framework for a reduction of power of idle or partially loaded CPUs 20 LEO (learning for energy optimization) a framework based on a probabilistic graphical model for obtaining Pareto-optimal power and performance trade-offs 30 A framework implementing two EDP-optimizing (energy delay product) algorithms: SEA and SPRA 31 An extension to SLURM scheduler to implement a "uniform frequency" in different configuration modes 10 Power capping CoPPer framework using power capping and adaptive control to approximate non-linearities in the power and performance relationship 32 PShifter: dynamic redistribution of power budged between cluster nodes using power limitation for faster processes 33 Application optimizations A framework modeling impact of optimization and providing recommendations for energy savings 24 Preparing best application configuration and settings on a GPU 25 Controlling CPU frequency, disk spinning and network speed scaling 34 Hybrids of the above A software/hardware approach with power capping based on a framework that makes decisions on configurations going through nodes 1 A reinforcement learning framework using power capping and uncore frequency scaling for optimization of the power consumption and run time 35 Scheduling/software as well as resource management with the use of RAPL 11 Scheduling kernels within a GPU as well as frequency scaling 16 Subsequently, we provide a concise comparison of selected approaches presented in respective research works to energy-performance oriented optimization in high-performance computing and presentation of the contribution and differences presented by us within this article.…”