Multi-Agent Deep Reinforcement Learning-Based Resource Allocation in HPC/AI Converged Cluster

Abstract

As the complexity of deep learning (DL) networks and training data grows enormously, methods that scale with computation are becoming the future of artificial intelligence (AI) development. In this regard, the interplay between machine learning (ML) and high-performance computing (HPC) is an innovative paradigm to speed up the efficiency of AI research and development. However, building and operating an HPC/AI converged system require broad knowledge to leverage the latest computing, networking, and storage technologies. Moreover, an HPC-based AI computing environment needs an appropriate resource allocation and monitoring strategy to efficiently utilize the system resources. In this regard, we introduce a technique for building and operating a high-performance AI-computing environment with the latest technologies. Specifically, an HPC/AI converged system is configured inside Gwangju Institute of Science and Technology (GIST), called GIST AI-X computing cluster, which is built by leveraging the latest Nvidia DGX servers, high-performance storage and networking devices, and various open source tools. Therefore, it can be a good reference for building a small or middlesized HPC/AI converged system for research and educational institutes. In addition, we propose a resource allocation method for DL jobs to efficiently utilize the computing resources with multi-agent deep reinforcement learning (mDRL). Through extensive simulations and experiments, we validate that the proposed mDRL algorithm can help the HPC/AI converged cluster to achieve both system utilization and power consumption improvement. By deploying the proposed resource allocation method to the system, total job completion time is reduced by around 20% and inefficient power consumption is reduced by around 40%.

Integrating HPC, AI, and Workflows for Scientific Data Analysis: Report from Dagstuhl Seminar 23352

“…• Resource Allocation: AI optimizes resource utilization in HPC clusters by dynamically (Chien, Lai, and Chao 2019) allocating computing resources based on workload demands or previous runs (Narantuya et al 2022). This leads to cost savings and improved efficiency in resource usage.…”

Systematic Review on Deep Reinforcement Learning-Based Energy Management for Different Building Types

“…One unique study implemented by Narantuya et al utilized a multi-agent DRL (mDRL) based on a DQN to optimize computational resource allocation in high-performance computing (HPC)/AI systems. Their system was further deployed in real-time, reducing the task completion time by 20% and the energy consumption by 40% [119]. Finally, Beimann et al conducted a comparative analysis of four different DRL methods for the control of a simulated HVAC system of a data centre.…”

Container Allocation in Cloud Environment Using Multi-Agent Deep Reinforcement Learning

“…Reinforcement learning (RL) has been recently adopted to solve cloud and edgecomputing resource allocation problems [4][5][6][7][8][9], and specifically container placement [10,11]. Busoniu et al [12] presented a comprehensive survey of multi-agents where the agents are capable of discovering a solution on their own using reinforcement learning.…”