Parallel I/O Performance for Application-Level Checkpointing on the Blue Gene/P System

Fu, Jing; Min, Misun; Latham, Robert; Carothers, Christopher D.

doi:10.1109/cluster.2011.81

Cited by 14 publications

(11 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the initiator receives all the checkpoints files, it sends these files to the stable memory. The initiator can use I/O strategies such as data sieving and collective I/O [26] to improve the storage time.…”

Section: Resynchronization Phasementioning

confidence: 99%

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

Meroufel¹,

Belalem²

2014

SCPE

View full text Add to dashboard Cite

Abstract. Cloud computing is a new benchmark towards enterprise application development that can facilitate the execution of workflows in business process management system. The workflow technology can manage the business processes efficiently satisfying the requirements of modern enterprises. Besides the scheduling, the fault tolerance is a very important issue in the workflow management. In this paper, we analyse and compare between some existing checkpointing strategies, then we propose a lightweight checkpointing adequate to the cloud computing and the workflows characteristics. The proposed strategy is an Adaptive Time based Coordinated Checkpointing ATCCp, it ensures a strong consistency without any synchronization. ATCCp uses the concept of soft checkpointing to minimize the storage time and it uses the VIOLIN topology to improve the checkpointing performances. According to the experimental results, our approach decreases the overhead and the SLA violations.

show abstract

Section: Resynchronization Phasementioning

confidence: 99%

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

Meroufel¹,

Belalem²

2014

SCPE

View full text Add to dashboard Cite

show abstract

“…This functionality is handled by the FileIO class. OpenSBLI adopts the HDF5 format [25,26] as it features parallel read/write capabilities and therefore has the potential to overcome the serial input/output bottleneck currently plaguing many large-scale parallel applications [27,28]. Future releases of OpenSBLI will come with the ability to read in mesh files and the state fields from an HDF5 file, enabling the restarting of simulations from 'checkpoints' as well as the assignment of initial conditions that cannot be simply defined by a formula.…”

Section: Input and Outputmentioning

confidence: 99%

OpenSBLI: A framework for the automated derivation and parallel execution of finite difference solvers on a range of computer architectures

Jacobs

Jammy

Sandham

2017

Journal of Computational Science

View full text Add to dashboard Cite

Exascale computing will feature novel and potentially disruptive hardware architectures. Exploiting these to their full potential is non-trivial. Numerical modelling frameworks involving finite difference methods are currently limited by the 'static' nature of the hand-coded discretisation schemes and repeatedly may have to be re-written to run efficiently on new hardware. In contrast, OpenSBLI uses code generation to derive the model's code from a high-level specification. Users focus on the equations to solve, whilst not concerning themselves with the detailed implementation. Source-to-source translation is used to tailor the code and enable its execution on a variety of hardware.

show abstract

“…Many such studies differ from ours in that they utilize overall performance metrics from multiple application runs at different scales in order to characterize I/O system performance. Often, when a new type of supercomputer becomes available, HPC researchers perform tests to determine its exact performance and behavior under certain loads [3], [4]. Other research studies focus on understanding parallel I/O forwarding software runs on compute and I/O nodes, bridges networks, and provides aggregation of independent I/O.…”

Section: Related Workmentioning

confidence: 99%

A Visual Network Analysis Method for Large-Scale Parallel I/O Systems

Sigovan

Muelder

et al. 2013

2013 IEEE 27th International Symposium on Parallel and Distributed Processing

View full text Add to dashboard Cite

Abstract-Parallel applications rely on I/O to load data, store end results, and protect partial results from being lost to system failure. Parallel I/O performance thus has a direct and significant impact on application performance. Because supercomputer I/O systems are large and complex, one cannot directly analyze their activity traces. While several visual or automated analysis tools for large-scale HPC log data exist, analysis research in the high-performance computing field is geared toward computation performance rather than I/O performance. Additionally, existing methods usually do not capture the network characteristics of HPC I/O systems. We present a visual analysis method for I/O trace data that takes into account the fact that HPC I/O systems can be represented as networks. We illustrate performance metrics in a way that facilitates the identification of abnormal behavior or performance problems. We demonstrate our approach on I/O traces collected from existing systems at different scales.

show abstract

Parallel I/O Performance for Application-Level Checkpointing on the Blue Gene/P System

Cited by 14 publications

References 16 publications

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

OpenSBLI: A framework for the automated derivation and parallel execution of finite difference solvers on a range of computer architectures

A Visual Network Analysis Method for Large-Scale Parallel I/O Systems

Contact Info

Product

Resources

About