Motivation and Implementation of a Dynamic Remote Storage System for I/O Demanding HPC Applications

Neuer, Matthias; Salk, Juergen; Berger, Holger; Focht, Erich; Mosch, Christian; Siegmund, Karsten; Kushnarenko, Volodymyr; Kombrink, Stefan; Wesner, Stefan

doi:10.1007/978-3-319-46079-6_42

Cited by 2 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several basic methods. A Dynamic Remote Scratch [22] implementation was developed to create an on-demand block device and use it with local SSDs as a LVM [23] device. Another software based solution is the RAMDISK Storage Accelerator [24].…”

Section: System Reconfigurationmentioning

confidence: 99%

Ad-Hoc File Systems At Extreme Scales

Soysal

Streit

2021

High Performance Computing in Science and Engineering '19

View full text Add to dashboard Cite

This work presents the results of the project with the acronym ADA-FS (Advanced Data Placement via Ad-hoc File Systems at extreme scale). The project which has been approved for the ForHLR II aims to improve I/O performance for highly parallel applications by using distributed on-demand file systems. These temporary file systems are created on the allocated compute nodes, using the node-local disks for the on-demand file system. Through integration into the scheduling system of the supercomputer, it can be requested like any other resource. The research approach contains the design of the file system itself as well as the questions about the right planning strategy for the necessary I/O transfers. In the granted project for the ForHLR II we are investigating the methods on how to integrate the approach into a HPC system. Also, we are evaluating the impact of the on-demand created file systems to running HPC jobs and the applications.

show abstract

Section: System Reconfigurationmentioning

confidence: 99%

Ad-Hoc File Systems At Extreme Scales

Soysal

Streit

2021

High Performance Computing in Science and Engineering '19

View full text Add to dashboard Cite

show abstract

“…There are several basic methods. A Dynamic Remote Scratch [23] implementation was developed to create an on-demand block device and use it with local SSDs as a LVM [24] device. This approach relies on a dedicated storage system.…”

Section: Related Workmentioning

confidence: 99%