A high performance redundancy scheme for cluster file systems

Pillai, M.; Lauria, Mario

doi:10.1504/ijhpcn.2004.008895

Cited by 5 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overview Redundant computation, process replication, and state machine replication have long histories and have been used extensively in distributed [13,24,25,27,43,47,79,81,84,85,158,170,195], mission critical [18,130,144,170], and storage systems [101,102,147,148] as a technique to improve fault tolerance. In state machine replication, one or more replicas of each process is maintained and every node computes deterministically in response to a given external input, for example a message being received.…”

Section: State Machine Replicationmentioning

confidence: 99%

Keeping checkpoint/restart viable for exascale systems.

Riesen¹,

Bridges

Stearley³

et al. 2011

View full text Add to dashboard Cite

Next-generation exascale systems, those capable of performing a quintillion (10 18 ) operations per second, are expected to be delivered in the next 8-10 years. These systems, which will be 1,000 times faster than current systems, will be of unprecedented scale. As these systems continue to grow in size, faults will become increasingly common, even over the course of small calculations. Therefore, issues such as fault tolerance and reliability will limit application scalability. Current techniques to ensure progress across faults like checkpoint/restart, the dominant fault tolerance mechanism for the last 25 years, are increasingly problematic at the scales of future systems due to their excessive overheads. In this work, we evaluate a number of techniques to decrease the overhead of checkpoint/restart and keep this method viable for future exascale systems. More specifically, this work evaluates state-machine replication to dramatically increase the checkpoint interval (the time between successive checkpoint) and hash-based, probabilistic incremental checkpointing using graphics processing units to decrease the checkpoint commit time (the time to save one checkpoint). Using a combination of empirical analysis, modeling, and simulation, we study the costs and benefits of these approaches on a wide range of parameters. These results, which cover of number of high-performance computing capability workloads, different failure distributions, hardware mean time to failures, and I/O bandwidths, show the potential benefits of these techniques for meeting the reliability demands of future exascale platforms.

show abstract

Section: State Machine Replicationmentioning

confidence: 99%

Keeping checkpoint/restart viable for exascale systems.

Riesen¹,

Bridges

Stearley³

et al. 2011

View full text Add to dashboard Cite

show abstract

CEFT: A cost-effective, fault-tolerant parallel virtual file system

Zhu

Jiang

2006

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

The vulnerability of computer nodes due to component failures is a critical issue for cluster-based file systems. This paper studies the development and deployment of mirroring in cluster-based parallel virtual file systems to provide fault tolerance and analyzes the tradeoffs between the performance and the reliability in the mirroring scheme. It presents the design and implementation of CEFT, a scalable RAID-10 style file system based on PVFS, and proposes four novel mirroring protocols depending on whether the mirroring operations are server-driven or client-driven, whether they are asynchronous or synchronous. The comparisons of their write performances, measured in a real cluster, and their reliability and availability, obtained through analytical modeling, show that these protocols strike different tradeoffs between the reliability and performance. Protocols with higher peak write performance are less reliable than those with lower peak write performance, and vice versa. A hybrid protocol is proposed to optimize this tradeoff.

show abstract

Building Strongly-Reliable and High-Performance Storage Cluster with Network Attached Storage Disks Using PCs

CaiBin

Xie

2005

The Fifth International Conference on Computer and Information Technology (CIT'05)

View full text Add to dashboard Cite

A high performance redundancy scheme for cluster file systems

Cited by 5 publications

References 15 publications

Keeping checkpoint/restart viable for exascale systems.

Keeping checkpoint/restart viable for exascale systems.

CEFT: A cost-effective, fault-tolerant parallel virtual file system

Building Strongly-Reliable and High-Performance Storage Cluster with Network Attached Storage Disks Using PCs

Contact Info

Product

Resources

About