Topology-aware resource management for HPC applications

Georgiou, Yiannis; Jeannot, Emmanuel; Mercier, Guillaume; Villiermet, Adèle

doi:10.1145/3007748.3007768

Cited by 13 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Solution ID Processors Memories 0 1 4 3 3 3 3 3 3 3 3 0 3 0 0 0 1 4 1 3 3 3 3 3 3 3 3 1 6 3 3 3 to determine the best choice among the available nodes based upon their position within the network [13], or emphasizing various targets, such as power-awareness [22] or resilience-awareness [5]. More recently, resource management has focused on the specific type of applications, such as MapReduce-based applications.…”

Section: Ldpc Ifs Solution Id Processors Memoriesmentioning

confidence: 99%

Predictive Resource Management for Next-Generation High-Performance Computing Heterogeneous Platforms

Massari

Pupykina

Agosta

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

High-Performance Computing (HPC) is rapidly moving towards the adoption of nodes characterized by an heterogeneous set of processing resources. This has already shown benefits in terms of both performance and energy efficiency. On the other side, heterogeneous systems are challenging from the application development and the resource management perspective. In this work, we discuss some outcomes of the MANGO project, showing the results of the execution of real applications on a emulated deeply heterogeneous systems for HPC. Moreover, we assessed the achievements of a proposed resource allocation policy, aiming at identifying a priori the best resource allocation options for a starting application.

show abstract

Section: Ldpc Ifs Solution Id Processors Memoriesmentioning

confidence: 99%

Predictive Resource Management for Next-Generation High-Performance Computing Heterogeneous Platforms

Massari

Pupykina

Agosta

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Some steps have been taken towards integrating more knowledge about the communication patterns of applications into batch schedulers. For instance, Georgiou et al studied the integration of TreeMatch into SLURM [9]. Given the communication matrix of an application, the scheduler minimizes the load of the network links by smartly mapping the application's processes on the resources.…”

Section: Related Workmentioning

confidence: 99%

Interference-Aware Scheduling Using Geometric Constraints

Bleuse

Dogeas

Lucarelli

et al. 2018

Euro-Par 2018: Parallel Processing

View full text Add to dashboard Cite

The large scale parallel and distributed platforms produce a continuously increasing amount of data which have to be stored, exchanged and used by various jobs allocated on different nodes of the platform. The management of this huge communication demand is crucial for the performance of the system. Meanwhile, we have to deal with more interferences as the trend is to use a single all-purpose interconnection network. In this paper, we consider two different types of communications: the flows induced by data exchanges during computations and the flows related to Input/Output operations. We propose a general model for interference-aware scheduling, where explicit communications are replaced by external topological constraints. Specifically, we limit the interferences of both communication types by adding geometric constraints on the allocation of jobs into machines. The proposed constraints reduce implicitly the data movements by restricting the set of possible allocations for each job. We present this methodology on the case study of simple network topologies, namely the line and the ring. We propose theoretical lower and upper bounds under different assumptions with respect to the platform and jobs characteristics. The obtained results illustrate well the difficulty of the problem even on simple topologies.

show abstract

“…Some steps have been taken towards integrating more knowledge about the communication patterns of applications into the batch scheduler. For example, Georgiou et al studied the integration of TreeMatch into SLURM [19]. Given the communication matrix of an application, the scheduler minimizes the load of the network links by smartly mapping the application's processes on the resources.…”

Section: Related Workmentioning

confidence: 99%

A Methodology for Handling Data Movements by Anticipation: Position Paper

Bleuse

Lucarelli

Trystram

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The enhanced capabilities of large scale parallel and distributed platforms produce a continuously increasing amount of data which have to be stored, exchanged and used by various tasks allocated on dierent nodes of the system. The management of such a huge communication demand is crucial for reaching the best possible performance of the system. Meanwhile, we have to deal with more interferences as the trend is to use a single all-purpose interconnection network whatever the interconnect (tree-based hierarchies or topology-based heterarchies). There are two dierent types of communications, namely, the ows induced by data exchanges during the computations, and the ows related to Input/Output operations. We propose in this paper a general model for interference-aware scheduling, where explicit communications are replaced by external topological constraints. Specically, the interferences of both communication types are reduced by adding geometric constraints on the allocation of tasks into machines. The proposed constraints reduce implicitly the data movements by restricting the set of possible allocations for each task. This methodology has been proved to be ecient in a recent study for a restricted interconnection network (a line/ring of processors which is an intermediate between a tree and higher dimensions grids/torus). The obtained results illustrated well the diculty of the problem even on simple topologies, but also provided a pragmatic greedy solution, which was assessed to be ecient by simulations. We are currently extending this solution for more complex topologies. This work is a position paper which describes the methodology, it does not focus on the solving part.

show abstract

Topology-aware resource management for HPC applications

Cited by 13 publications

References 23 publications

Predictive Resource Management for Next-Generation High-Performance Computing Heterogeneous Platforms

Predictive Resource Management for Next-Generation High-Performance Computing Heterogeneous Platforms

Interference-Aware Scheduling Using Geometric Constraints

A Methodology for Handling Data Movements by Anticipation: Position Paper

Contact Info

Product

Resources

About