BlobCR: Virtual disk based checkpoint-restart for HPC applications on IaaS clouds

Abstract

Infrastructure-as-a-Service (IaaS) cloud computing is gaining significant interest in industry and academia as an alternative platform for running HPC applications. Given the need to provide fault tolerance, support for suspend-resume and offline migration, an efficient Checkpoint-Restart mechanism becomes paramount in this context. We propose BlobCR, a dedicated checkpoint repository that is able to take live incremental snapshots of the whole disk attached to the virtual machine (VM) instances. BlobCR aims to minimize the performance overhead of checkpointing by persisting VM disk snapshots asynchronously in the background using a low overhead technique we call selective copy-on-write. It includes support for both application-level and process-level checkpointing, as well as support to roll back file system changes. Experiments at large scale demonstrate the benefits of our proposal both in synthetic settings and for a reallife HPC application.

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

“…However, we studied the ATCCp behavior compared to coordinated checkpointing CCp used in [1], [16], [35] and independent checkpointing ICp used in [15], [28], [36]. The goal is proofing that our approach ensures a strong consistency with the minimum cost (like the CCp) and with the minimum overhead (like the ICp).…”

“…The cloud provisions pools of computing resources as services via the internet using a pay-as-you-go price model that eliminates initial costly capital investments in hardware and infrastructure. Research and academic communities can leverage the benefit of the cloud price model for their computationintensive applications that traditionally run in HPC environments [34], [35], [36], [37] such as Amazon Web Services' HPC offering [38] or science cloud initiatives [39].…”

Transparent Throughput Elasticity for IaaS Cloud Storage Using Guest-Side Block-Level Caching

“…For example, a majority of high-performance computing (HPC) numerical simulations model the evolution of physical phenomena in time by using a bulk-synchronous approach. This involves a synchronization point at the end of each iteration in order to write intermediate output data about the simulation, as well as periodic checkpoints that are needed for a variety of tasks [5] such as migration, debugging, and minimizing the amount of lost computation in case of failures. Since many processes share the same storage (e.g., all processes on the same node share the same local disks), this behavior translates to periods of little I/O activity that are interleaved with periods of highly intensive I/O peaks.…”

High availability in clouds: systematic review and research challenges

“…In [24], authors propose BlobCR, a checkpoint framework for High Performance Computing (HPC) applications on IaaS. Their approach is directed at both application and process checkpoint levels through a distributed checkpoint repository.…”