Resource Management for Power-Constrained HEVC Transcoding Using Reinforcement Learning

Abstract

The advent of online video streaming applications and services along with the users' demand for high-quality contents require High Efficiency Video Coding (HEVC), which provides higher video quality and more compression at the cost of increased complexity. On one hand, HEVC exposes a set of dynamically tunable parameters to provide trade-offs among Quality-of-Service (QoS), performance, and power consumption of multi-core servers on the video providers' data center. On the other hand, resource management of modern multi-core servers is in charge of adapting system-level parameters, such as operating frequency and multithreading, to deal with concurrent applications and their requirements. Therefore, efficient multiuser HEVC streaming necessitates joint adaptation of application-and system-level parameters. Nonetheless, dealing with such a large and dynamic design space is challenging and difficult to address through conventional resource management strategies. Thus, in this work, we develop a multi-agent Reinforcement Learning framework to jointly adjust application-and system-level parameters at runtime to satisfy the QoS of multiuser HEVC streaming in power-constrained servers. In particular, the design space, composed of all design parameters, is split into smaller independent sub-spaces. Each design sub-space is assigned to a particular agent so that it can explore it faster, yet accurately. The benefits of our approach are revealed in terms of adaptability and quality (with up to to 4× improvements in terms of QoS when compared to a static resource management scheme), and learning time (6× faster than an equivalent mono-agent implementation). Finally, we show that the power-capping techniques formulated outperform the hardware-based power capping with respect to quality.

Applying Game-Learning Environments to Power Capping Scenarios via Reinforcement Learning

“…With the increasing interest on Machine Learning, RL has been proved as a valid alternative to tackle these scenarios [4,8], as it presents several advantages over the traditional approaches. Among others, a less deep knowledge of the problem is required to formulate the solution, as well as the ability to obtain input-independent policies.…”

“…As drawbacks, ML-based approaches can lead to long training periods, as well as to solutions that, being valid, are difficult to understand by a human compared with heuristics. However, the application of these techniques to resource management is far from being trivial, and typically requires ad-hoc implementations [4,8]. 4 Casting a power capping scenario with Gym…”

Averting BER Floor with Iterative Source and Channel Decoding for Layered Steered Space-Time Codes

“…An efficient algorithm adjustable to variations in power and altering network environments was presented for the H.265 video streams in [ 31 ]. To boost the quality of service for the compressed H.265 multi-user stream, reinforcement learning techniques were adopted in [ 32 ]. Table 1 shows an overview of the methods and systems investigated recently related to wireless video communication.…”