LACIO: A New Collective I/O Strategy for Parallel I/O Systems

Abstract

Abstract-Parallel applications benefit considerably from the rapid advance of processor architectures and the available massive computational capability, but their performance suffers from large latency of I/O accesses. The poor I/O performance has been attributed as a critical cause of the low sustained performance of parallel systems. Collective I/O is widely considered a critical solution that exploits the correlation among I/O accesses from multiple processes of a parallel application and optimizes the I/O performance. However, the conventional collective I/O strategy makes the optimization decision based on the logical file layout to avoid multiple file system calls and does not take the physical data layout into consideration. On the other hand, the physical data layout in fact decides the actual I/O access locality and concurrency. In this study, we propose a new collective I/O strategy that is aware of the underlying physical data layout. We confirm that the new Layout-Aware Collective I/O (LACIO) improves the performance of current parallel I/O systems effectively with the help of noncontiguous file system calls. It holds promise in improving the I/O performance for parallel systems.

A Checkpoint of Research on Parallel I/O for High-Performance Computing

“…Their proposal consists of rearranging the partition of the file domain and of requests from aggregators, aiming at matching the physical layout on the servers. In a more recent publication by Chen et al [13], they use the physical data layout information in their collective I/O approach so each aggregator will access as few servers as possible. McLay et al [69] demonstrate choosing the appropriate stripe size is critical to collective write performance.…”

Iteration Based Collective I/O Strategy for Parallel I/O Systems

“…There has been significant amount of research studies about techniques to improve the parallel I/O performance of data-intensive scientific applications [19,[24][25][26][27]. A lot of researches aim to design and evaluate efficient and scalable I/O strategies to support I/O middleware concepts such as the file data description and collective I/O.…”

“…Chen et al [25] propose a new collective I/O strategy called Layout Aware Collective I/O (LACIO), which makes the optimization decision based on the physical data layout information. The proposed strategy rearranges the partitions of file domains and the requests of aggregators in a fashion that match the physical data layout on storage servers of a system in order to reduce access contentions and exploit better concurrency and locality.…”

Memory-conscious collective I/O for extreme scale HPC systems

“…[24]. A layout-aware collective I/O strategy was recently introduced in [25] to provide a better integration of parallel I/O middleware and parallel file system and improve the overall performance. These strategies collect and merge small and noncontiguous requests into a large and contiguous one for aggregators to carry out more efficiently, or ship I/O calls to dedicated processes.…”