Efficient Dynamic Pinning of Parallelized Applications by Distributed Reinforcement Learning

Abstract

Abstract. This paper introduces a resource allocation framework specifically tailored for addressing the problem of dynamic placement (or pinning) of parallelized applications to processing units. Under the proposed setup each thread of the parallelized application constitutes an independent decision maker (or agent), which (based on its own prior performance measurements and its own prior CPU-affinities) decides on which processing unit to run next. Decisions are updated recursively for each thread by a resource manager/scheduler which runs in parallel to the application's threads and periodically records their performances and assigns to them new CPU affinities. For updating the CPU-affinities, the scheduler uses a distributed reinforcement-learning algorithm, each branch of which is responsible for assigning a new placement strategy to each thread. According to this algorithm, prior allocations are going to be reinforced in the future proportionally to their prior performance. The proposed resource allocation framework is flexible enough to address alternative optimization criteria, such as maximum average processing speed and minimum speed variance among threads. We demonstrate analytically that convergence to locally-optimal placements is achieved asymptotically. Finally, we validate these results through experiments in Linux platforms.

Efficient Dynamic Pinning of Parallelized Applications by Reinforcement Learning with Applications

“…This work extends prior work of the authors [8] in two directions: (a) we introduce a new type of reinforcement-learning dynamics that admits faster adjustment towards better allocations, and (b) evaluation is performed over a real-world application, that is a parallelized implementation of the Ant-Colony Optimization metaheuristic.…”

“…In comparison to [8], the difference lies in the reinforcement direction. As Equation (4) dictates, the strategy vector is only adjusted when a performance is higher than the running-average performanceū i , which provides a faster adjustment towards better assignments.…”

“…The convergence properties of this class of dynamics can be derived following the exact same reasoning used for the learning dynamics presented in [8]. In fact, it can be shown that the dynamics approach asymptotically a set of allocations which includes the solutions of the centralized optimization (1).…”

Learning-Based Dynamic Pinning of Parallelized Applications in Many-Core Systems

“…Recognizing this need for both learning-and distributedbased optimization, and contrary to the aforementioned references on pinning of parallelized applications, our earlier work [6], [7] proposed a scheduling scheme for optimally allocating threads of a parallelized application that combines both a learning-and a distributed-based optimization. It requires a minimum information exchange, where only measurements collected from each running thread are needed.…”

“…In our previous work [6], [7], we have proposed a reinforcement-learning-based distributed scheduling framework (PaRLSched), adapted to Uniform Memory Architectures (UMA). In this paper, our goal is to provide a generalized methodology that also extends to Non-Uniform Memory Architectures (NUMA).…”