Ad-Hoc File Systems At Extreme Scales

Abstract

This work presents the results of the project with the acronym ADA-FS (Advanced Data Placement via Ad-hoc File Systems at extreme scale). The project which has been approved for the ForHLR II aims to improve I/O performance for highly parallel applications by using distributed on-demand file systems. These temporary file systems are created on the allocated compute nodes, using the node-local disks for the on-demand file system. Through integration into the scheduling system of the supercomputer, it can be requested like any other resource. The research approach contains the design of the file system itself as well as the questions about the right planning strategy for the necessary I/O transfers. In the granted project for the ForHLR II we are investigating the methods on how to integrate the approach into a HPC system. Also, we are evaluating the impact of the on-demand created file systems to running HPC jobs and the applications.