MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS

Prost, Jean-Pierre; Treumann, Richard; Hedges, Richard; Jia, Bin; Koniges, Alice

doi:10.1145/582034.582051

Cited by 60 publications

(41 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I/O forwarding has also been used in Prost et al (2001), Oldfield et al (2006), Nisar et al (2008), Fu et al (2010), Docan et al (2010) and May (2001) to reduce the I/O impact on computing. The IBM Blue Gene series of supercomputers (Yu et al, 2006) uses independent I/O nodes in their system to handle I/O requests, which are generated in computer nodes and forwarded to I/O nodes.…”

Section: Related Workmentioning

confidence: 99%

A fast input/output library for high-resolution climate models

Wang

Yang

et al. 2014

Geosci. Model Dev.

View full text Add to dashboard Cite

Abstract. We describe the design and implementation of climate fast input/output (CFIO), a fast input/output (I/O) library for high-resolution climate models. CFIO provides a simple method for modelers to overlap the I/O phase with the computing phase automatically, so as to shorten the running time of numerical simulations. To minimize the code modifications required for porting, CFIO provides similar interfaces and features to parallel Network Common Data Form (PnetCDF), which is one of the most widely used I/O libraries in climate models. We deployed CFIO in three highresolution climate models, including two ocean models (POP and LICOM) and one sea ice model (CICE). The experimental results show that CFIO improves the performance of climate models significantly versus the original serial I/O approach. When running with CFIO at 0.1 • resolution with about 1000 CPU cores, we managed to reduce the running time by factors of 7.9, 4.6 and 2.0 for POP, CICE, and LI-COM, respectively. We also compared the performance of CFIO against two existing libraries, PnetCDF and parallel I/O (PIO), in different scenarios. For scenarios with both data output and computations, CFIO decreases the I/O overhead compared to PnetCDF and PIO.

show abstract

Section: Related Workmentioning

confidence: 99%

A fast input/output library for high-resolution climate models

Wang

Yang

et al. 2014

Geosci. Model Dev.

View full text Add to dashboard Cite

show abstract

“…We note that besides PnetCDF, there are several general purpose parallel file systems such as GPFS on IBM platforms, the Lustre file system on Cray platforms, and parallel virtual file systems (PVFS) developed at ANL (e.g. Carns et al 2000;Prost et al 2001).…”

Section: Community Atmosphere Model (Cam)mentioning

confidence: 99%

Efficient Parallel I/O in Community Atmosphere Model (CAM)

Tseng

Ding

2008

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Century-long global climate simulations at high resolutions generate large amounts of data in a parallel architecture. Currently, the community atmosphere model (CAM), the atmospheric component of the NCAR community climate system model (CCSM), uses sequential I/O which causes a serious bottleneck for these simulations. We describe the parallel I/O development of CAM in this paper. The parallel I/ O combines a novel remapping of 3-D arrays with the parallel netCDF library as the I/O interface. Because CAM history variables are stored in disk file in a different index order than the one in CPU resident memory because of parallel decomposition, an index reshuffle is done on the fly. Our strategy is first to remap 3-D arrays from its native decomposition to zdecomposition on a distributed architecture, and from there write data out to disk. Because z-decomposition is consistent with the last array dimension, the data transfer can occur at maximum block sizes and, therefore, achieve maximum I/ O bandwidth. We also incorporate the recently developed parallel netCDF library at Argonne/Northwestern as the collective I/O interface, which resolves a long-standing issue because netCDF data format is extensively used in climate system models. Benchmark tests are performed on several platforms using different resolutions. We test the performance of our new parallel I/O on five platforms (SP3, SP4, SP5, Cray X1E, BlueGene/L) up to 1024 processors. More than four realistic model resolutions are examined, e.g. EUL T85 (~1.4°), FV-B (2° × 2.5°), FV-C (1° × 1.25°), and FV-D (0.5° × 0.625°) resolutions. For a standard single history output of CAM 3.1 FV-D resolution run (multiple 2-D and 3-D arrays with total size 4.1 GB), our parallel I/O speeds up by a factor of 14 on IBM SP3, compared with the existing I/O; on IBM SP5, we achieve a factor of 9 speedup. The estimated time for a typical century-long simulation of FV D-resolution on IBM SP5 shows that the I/O time can be reduced from more than 8 days (wall clock) to less than 1 day for daily output. This parallel I/O is also implemented on IBM BlueGene/ L and the results are shown, whereas the existing sequential I/O fails due to memory usage limitation.

show abstract

“…IBM's GPFS [31], which is POSIX compliant, has an optimized MPI-IO implementation [28] for improving collective I/O access. It has a global lock manager which hands out "lock tokens" to clients, which helps with optimizing client-side caching.…”

Section: History and Our Dlm Approachmentioning

confidence: 99%

Noncontiguous locking techniques for parallel file systems

Ching

Liao

Choudhary

et al. 2007

Proceedings of the 2007 ACM/IEEE Conference on Supercomputing

View full text Add to dashboard Cite

Many parallel scientific applications use high-level I/O APIs that offer atomic I/O capabilities. Atomic I/O in current parallel file systems is often slow when multiple processes simultaneously access interleaved, shared files. Current atomic I/O solutions are not optimized for handling noncontiguous access patterns because current locking systems have a fixed file system block-based granularity and do not leverage highlevel access pattern information.In this paper we present a hybrid lock protocol that takes advantage of new list and datatype byte-range lock description techniques to enable high performance atomic I/O operations for these challenging access patterns. We implement our scalable distributed lock manager (DLM) in the PVFS parallel file system and show that these techniques improve locking throughput over a naive noncontiguous locking approach by several orders of magnitude in an array of lockonly tests. Additionally, in two scientific I/O benchmarks, we show the benefits of avoiding false sharing with our byterange granular DLM when compared against a block-based lock system implementation.

show abstract

MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS

Cited by 60 publications

References 3 publications

A fast input/output library for high-resolution climate models

A fast input/output library for high-resolution climate models

Efficient Parallel I/O in Community Atmosphere Model (CAM)

Noncontiguous locking techniques for parallel file systems

Contact Info

Product

Resources

About