Modeling Parallel Scientific Applications through their Input/Output Phases

Abstract

The increase in computational power in processing units and the complexity of scientific applications that use high performance computing require more efficient Input/Output (I/O) systems. To use the I/O systems more efficiently it is necessary to know its performance capacity to determine if it fulfills applications' I/O requirements. Evaluating the I/O system performance capacity is difficult due to the diversity of I/O architectures and the complexity of its I/O software stack. Furthermore, parallel scientific applications have different behavior depending on their access patterns. Then, it is necessary to have some method to evaluate the I/O subsystem capacity taking into account the applications access patterns without executing the application in each I/O subsystem.Here, we propose a methodology to evaluate the I/O subsystem performance capacity through an I/O model of the parallel application independent of the I/O subsystem. This I/O model is composed of I/O phases representing "where" and "when" the I/O operations are performed into application logic.This approach encompasses the I/O subsystem evaluation at I/O library level for the application I/O model. The I/O phases are replicated by benchmark IOR which is executed in the target subsystem. This approach was used to estimate the I/O time of an application in different subsystems. The results show an relative error of estimation lower than 10%. This approach was also utilized to select the I/O subsystem that provide less I/O time for the application.

Techniques for modeling large-scale HPC I/O workloads

“…In general, synthesizing representative I/O workloads requires some sort of model or characterization of the target application's I/O behavior. In most existing research, HPC application I/O characterizations are generated by first tracing the I/O operations of a target application and performing an in-depth analysis of this trace offline [5,14,18,30,37]. In contrast, Darshan I/O characterizations are automatically generated at runtime.…”

Tools for Analyzing Parallel I/O

“…PIOM-MPPIOM-MP (formerly known as PAS2P-IO[15,16]) represents the MPI application's I/O behavior by using I/O phases. A phase is a consecutive sequence of similar access patterns into the logical view of a file.…”