High Performance File I/O for The Blue Gene/L Supercomputer

Yu, Hao; Sahoo, Ramendra K.; Howson, Christopher; Almási, George; Castaños, José G.; Gupta, Manish; Moreira, José E.; Parker, Jeff; Engelsiepen, Thomas; Ross, Robert; Thakur, Rajeev; Latham, Robert; Gropp, William

doi:10.1109/hpca.2006.1598125

Cited by 35 publications

(26 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This gives Equation (1) where tts is the time to solve, tto is the time to output and ttt is the time to transfer. Expanding Equation (1), we obtain the constraint specified in Equation (2).…”

Section: B Optimization Methodsmentioning

confidence: 99%

“…Simulations running on thousands of cores take less than a second of execution time per time step [1]. Parallel I/O can enable very high I/O bandwidth of the range of 5 -20 GBps on large number of cores [2], [3]. A combination of high simulation rate and high I/O bandwidth leads to high rate of generation of gigabytes of climate data as output and hence rapid accumulation of data in the stable storage.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Adaptive Framework for Simulation and Online Remote Visualization of Critical Climate Applications in Resource-constrained Environments

Malakar

Natarajan

Vadhiyar

2010

2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Abstract-Critical climate applications like cyclone tracking and earthquake modeling require high-performance simulations and online visualization simultaneously performed with the simulations for timely analysis. Remote visualization of critical climate events enables joint analysis by geographically distributed climate science community. However, resource constraints including limited storage and slow networks can limit the effectiveness of such online visualization. In this work, we have developed an adaptive framework that simultaneously performs numerical simulations and online remote visualization of critical climate applications in resource-constrained environments. Our framework considers both application and resource dynamics to adapt various application and resource parameters including simulation resolutions, resource configurations and amount of data for visualization. We have developed two algorithms for processor allocation for simulations and the frequency of data for visualization. We show that our optimization method is able to provide about 30% higher simulation rate and consumes about 25-50% lesser storage space than the greedy approach.

show abstract

Section: B Optimization Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Adaptive Framework for Simulation and Online Remote Visualization of Critical Climate Applications in Resource-constrained Environments

Malakar

Natarajan

Vadhiyar

2010

2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…Of particular importance for strong scaling algorithms, MPI-IO can aggregate access to shared files in order to mitigate the effect of smaller writes at scale. Collective MPI-IO routines also introduce precise block alignment, which has been shown to be an important factor in GPFS performance [15], [12]. Figure 12 shows the per node I/O performance achieved by all three versions of the test application with two checkpoints.…”

Section: Application Tuning Case Studymentioning

confidence: 99%

Understanding and Improving Computational Science Storage Access through Continuous Characterization

et al. 2011

View full text Add to dashboard Cite

Abstract-Computational science applications are driving a demand for increasingly powerful storage systems. While many techniques are available for capturing the I/O behavior of individual application trial runs and specific components of the storage system, continuous characterization of a production system remains a daunting challenge for systems with hundreds of thousands of compute cores and multiple petabytes of storage. As a result, these storage systems are often designed without a clear understanding of the diverse computational science workloads they will support.In this study, we outline a methodology for scalable, continuous, systemwide I/O characterization that combines storage device instrumentation, static file system analysis, and a new mechanism for capturing detailed application-level behavior. This methodology allows us to quantify systemwide trends such as the way application behavior changes with job size, the "burstiness" of the storage system, and the evolution of file system contents over time. The data also can be examined by application domain to determine the most prolific storage users and also investigate how their I/O strategies correlate with I/O performance. At the most detailed level, our characterization methodology can also be used to focus on individual applications and guide tuning efforts for those applications.We demonstrate the effectiveness of our methodology by performing a multilevel, two-month study of Intrepid, a 557-teraflop IBM Blue Gene/P system. During that time, we captured application-level I/O characterizations from 6,481 unique jobs spanning 38 science and engineering projects with up to 163,840 processes per job. We also captured patterns of I/O activity in over 8 petabytes of block device traffic and summarized the contents of file systems containing over 191 million files. We then used the results of our study to tune example applications, highlight trends that impact the design of future storage systems, and identify opportunities for improvement in I/O characterization methodology.

show abstract

“…Some systems limit the number of parallel file system clients that an application can access [14] [15]. This approach allows an application to always get some bandwidth through its allocated clients.…”

Section: A Limitations Of Parallel Storage Managementmentioning

confidence: 99%

vPFS: Bandwidth virtualization of parallel storage systems

Arteaga

Zhao

et al. 2012

012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST)

View full text Add to dashboard Cite

Abstract-Existing parallel file systems are unable to differentiate I/Os requests from concurrent applications and meet per-application bandwidth requirements. This limitation prevents applications from meeting their desired Quality of Service (QoS) as high-performance computing (HPC) systems continue to scale up. This paper presents vPFS, a new solution to address this challenge through a bandwidth virtualization layer for parallel file systems. vPFS employs user-level parallel file system proxies to interpose requests between native clients and servers and to schedule parallel I/Os from different applications based on configurable bandwidth management policies. vPFS is designed to be generic enough to support various scheduling algorithms and parallel file systems. Its utility and performance are studied with a prototype which virtualizes PVFS2, a widely used parallel file system. Enhanced proportional sharing schedulers are enabled based on the unique characteristics (parallel striped I/Os) and requirement (high throughput) of parallel storage systems. The enhancements include new threshold-and layout-driven scheduling synchronization schemes which reduce global communication overhead while delivering total-service fairness. An experimental evaluation using typical HPC benchmarks (IOR, NPB BTIO) shows that the throughput overhead of vPFS is small (< 3% for write, < 1% for read). It also shows that vPFS can achieve good proportional bandwidth sharing (> 96% of target sharing ratio) for competing applications with diverse I/O patterns.

show abstract

High Performance File I/O for The Blue Gene/L Supercomputer

Abstract: Parallel I/O plays a crucial role for most data-intensive applications running on massively parallel systems like

Cited by 35 publications

References 12 publications

An Adaptive Framework for Simulation and Online Remote Visualization of Critical Climate Applications in Resource-constrained Environments

An Adaptive Framework for Simulation and Online Remote Visualization of Critical Climate Applications in Resource-constrained Environments

Understanding and Improving Computational Science Storage Access through Continuous Characterization

vPFS: Bandwidth virtualization of parallel storage systems

Contact Info

Product

Resources

About