Topology-aware job mapping

Abstract

A Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for e ciently delivering computing power to applications in supercomputing environments. Its main intelligence relies on resource selection techniques to find the most adapted resources to schedule the users' jobs. This paper introduces a new method that takes into account the topology of the machine and the application characteristics to determine the best choice among the available nodes of the platform, based upon the network topology and taking into account the applications communication pattern. To validate our approach, we integrate this algorithm as a plugin for Slurm, a well-known and widespread RJMS. We assess our plugin with di↵erent optimization schemes by comparing with the default topologyaware Slurm algorithm, using both emulation and simulation of a large-scale platform and by carrying out experiments in a real cluster. We show that transparently taking into account a job communication pattern and the topology allows for relevant performance gains.

Process mapping on any topology with TopoMatch

“…In this case, an optimization consists of performing a topology-aware selection of the nodes taking into account the communication pattern of the application and performing the best choice among the available resources. This can be done using Topo-Match [31] thanks to its constraints feature. It works as follows.…”

ROA-CONS: Raccoon Optimization for Job Scheduling

“…Many researchers have investigated the impact of job schedules on parallel computer performance [1][2][3][4]. These experiments have demonstrated that the superior job planning algorithm improves efficiency and performance.…”