From Static to Malleable: Improving Flexibility and Compatibility in Burst Buffer File Systems

Vef, Marc-André; Miranda, Alberto; Nou, Ramon; Brinkmann, André

doi:10.1007/978-3-031-40843-4_1

Cited by 1 publication

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Node-local Burst Buffer is located on each computing node. Node-local Burst Buffer has good scalability [18]. However, the deployment cost of node-local Burst Buffer architecture is high and it is not suitable for scenarios that require data sharing.…”

Section: Introductionmentioning

confidence: 99%

GABB: the plan-based job scheduling optimized by genetic algorithm for HPC systems with shared burst buffers

Lv,

Xie,

Song

2024

Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024)

View full text Add to dashboard Cite

Burst Buffer was proposed to address the growing performance gap between computation and I/O in high-performance computing systems (HPC). However, the introduction of Burst Buffer has brought new issues in system resource management and job scheduling. Resource management systems on HPC platforms lack efficient management of multiple types of resources, and the allocation of computing resources and Burst Buffer resources is independent of each other. This can lead to underutilization of system resources and job blocking. To address these issues, this paper proposes GABB, a plan-based job scheduling strategy for shared Burst Buffer, and utilizes genetic algorithms for optimization. GABB employs a plan-based job scheduling strategy to uniformly schedule all jobs in the waiting queue to generate an execution plan. Furthermore, GABB comprehensively considers changes in system resources and the job's demand for computing and Burst Buffer resources during the job scheduling process. Finally, GABB utilizes an improved genetic algorithm to optimize the job scheduling scheme. We conducted experimental simulations of a shared Burst Buffer system and implemented a plan-based job scheduling algorithm. The experimental results indicate that BBGA significantly reduces the mean waiting time of jobs by over 20% compared to the Shortest Job First (SJF) algorithm, and reduces the mean bounded slowdown of jobs by over 25%.

show abstract

Section: Introductionmentioning

confidence: 99%