Effects and Benefits of Node Sharing Strategies in HPC Batch Systems

Frank, Alvaro; Süß, Tim; Brinkmann, André

doi:10.1109/ipdps.2019.00016

Cited by 7 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All approaches to improve node-level efficiency are translated into cluster-level improvements by fitting batch schedulers like SLURM [53] with job co-scheduling capabilities that consider how node-level resources are shared [17,23,34,52,54,55]. The coscheduling problem is related to node-level efficiency: better alternatives to in-node resource sharing can be combined with existing scheduling policies to deliver better cluster-level efficiency.…”

Section: Introductionmentioning

confidence: 99%

nOS-V: Co-Executing HPC Applications Using System-Wide Task Scheduling

Álvarez¹,

Sala²,

Beltran³

2022

Preprint

View full text Add to dashboard Cite

Future Exascale systems will feature massive parallelism, manycore processors and heterogeneous architectures. In this scenario, it is increasingly difficult for HPC applications to fully and efficiently utilize the resources in system nodes. Moreover, the increased parallelism exacerbates the effects of existing inefficiencies in current applications. Research has shown that co-scheduling applications to share system nodes instead of executing each application exclusively can increase resource utilization and efficiency. Nevertheless, the current oversubscription and co-location techniques to share nodes have several drawbacks which limit their applicability and make them very application-dependent.This paper presents co-execution through system-wide scheduling. Co-execution is a novel fine-grained technique to execute multiple HPC applications simultaneously on the same node, outperforming current state-of-the-art approaches. We implement this technique in nOS-V, a lightweight tasking library that supports coexecution through system-wide task scheduling. Moreover, nOS-V can be easily integrated with existing programming models, requiring no changes to user applications. We showcase how co-execution with nOS-V significantly reduces schedule makespan for several applications on single node and distributed environments, outperforming prior node-sharing techniques.

show abstract

Section: Introductionmentioning

confidence: 99%

nOS-V: Co-Executing HPC Applications Using System-Wide Task Scheduling

Álvarez¹,

Sala²,

Beltran³

2022

Preprint

View full text Add to dashboard Cite

show abstract

Exploring Mapping Strategies for Co-allocated HPC Applications

Vardas,

Hunold,

Swartvagher

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In modern HPC systems with deep hierarchical architectures, large-scale applications often struggle to efficiently utilize the abundant cores due to the saturation of resources such as memory. Co-allocating multiple applications to share compute nodes can mitigate these issues and increase system throughput. However, co-allocation may harm the performance of individual applications due to resource contention. Past research suggests that topology-aware mappings can improve the performance of parallel applications that do not share resources. In this work, we implement application-oblivious, topology-aware process-to-core mappings via different core enumerations that support the co-allocation of parallel applications. We show that these mappings have a significant impact on the available memory bandwidth. We explore how these process-to-core mappings can affect the individual application duration as well as the makespan of job schedules when they are combined with co-allocation. Our main objective is to assess whether co-allocation with a topology-aware mapping can be a viable alternative to the exclusive node allocation policies that are currently common in HPC clusters.

show abstract