Improving cluster availability using workstation validation

Heath, Taliver; Martin, Richard P.; Nguyen, Thu D.

doi:10.1145/511361.511362

Cited by 23 publications

(45 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the large body of research already dedicated to modeling the availability of parallel and distributed computer systems-see [19,17,18,9] and the references within-, relatively little attention has been given to space-correlated errors and failures [19,4,11], despite their reported importance [8,17]. The main differences between this work and the previous work on space-correlated errors and failures is summarized in Table 12.…”

Section: Related Workmentioning

confidence: 98%

“…Since the scale and complexity of contemporary distributed systems make the occurrence of failures the rule rather than the exception, many fault tolerant resource management techniques have been designed recently [8,3,17]. The deployment of these techniques and the design of new ones depend on understanding the characteristics of failures in real systems.…”

Section: Introductionmentioning

confidence: 99%

“…A bursty arrival breaks an important assumption made by numerous fault tolerant algorithms [8,20,15], that of independent and identical distribution of failures among the components of the system. However, few studies [19,4,11] investigate the bursty arrival of failures for distributed systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Gallet¹,

Yigitbasi²,

Javadi³

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Distributed systems such as grids, peer-to-peer systems, and even Internet DNS servers have grown significantly in size and complexity in the last decade. This rapid growth has allowed distributed systems to serve a large and increasing number of users, but has also made resource and system failures inevitable. Moreover, perhaps as a result of system complexity, in distributed systems a single failure can trigger within a short time span several more failures, forming a group of time-correlated failures. To eliminate or alleviate the significant effects of failures on performance and functionality, the techniques for dealing with failures require good failure models. However, not many such models are available, and the available models are valid for few or even a single distributed system. In contrast, in this work we propose a model that considers groups of time-correlated failures and is valid for many types of distributed systems. Our model includes three components, the group size, the group inter-arrival time, and the resource downtime caused by the group. To validate this model, we use failure traces corresponding to fifteen distributed systems. We find that space-correlated failures are dominant in terms of resource downtime in seven of the fifteen studied systems. For each of these seven systems, we provide a set of model parameters that can be used in research studies or for tuning distributed systems. Last, as a result of our work six of the studied traces have been made available through the Failure Trace Archive

show abstract

Section: Related Workmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Gallet¹,

Yigitbasi²,

Javadi³

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Xu et al [40] performed a study of error logs collected from a heterogeneous distributed system consisting of 503 PC servers. Heath et al [13] considered failure data from three different clustered servers, ranging from 18 workstations to 89 workstations. Castillo et al [7], Iyer et al [15] and Meyer et al [24] have explored the effects of workload on different types of computer system failures.…”

Section: Failure Injectionmentioning

confidence: 99%

“…This includes the fitting of failure data to Weibull, lognormal and other specific distributions, each with different parameter settings, under the assumption of independent and identically distributed failures [20,19,13]. Other studies have demonstrated that the sequence of failures on some computer systems are correlated in various ways and that the failures tend to occur in bursts [37,36,40,28].…”

Section: Failure Injectionmentioning

confidence: 99%

Performance Implications of Failures in Large-Scale Cluster Scheduling

Zhang

Squillante

Sivasubramaniam

et al. 2005

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. As we continue to evolve into large-scale parallel systems, many of them employing hundreds of computing engines to take on mission-critical roles, it is crucial to design those systems anticipating and accommodating the occurrence of failures. Failures become a commonplace feature of such large-scale systems, and one cannot continue to treat them as an exception. Despite the current and increasing importance of failures in these systems, our understanding of the performance impact of these critical issues on parallel computing environments is extremely limited. In this paper we develop a general failure modeling framework based on recent results from large-scale clusters and then we exploit this framework to conduct a detailed performance analysis of the impact of failures on system performance for a wide range of scheduling policies. Our results demonstrate that such failures can have a significant impact on the mean job response time and mean job slowdown under existing scheduling policies that ignore failures. We therefore investigate different scheduling mechanisms and policies to address these performance issues. Our results show that periodic checkpointing of jobs seems to do little to ease this problem. On the other hand, we demonstrate that information about the spatial and temporal correlation of failure occurrences can be very useful in designing a scheduling (job allocation) strategy to enhance system performance, with the former providing the greatest benefits.

show abstract

Fault-Tolerance Techniques for High-Performance Computing

2015

Computer Communications and Networks

View full text Add to dashboard Cite

This report provides an introduction to resilience methods. The emphasis is on checkpointing, the de-facto standard technique for resilience in High Performance Computing. We present the main two protocols, namely coordinated checkpointing and hierarchical checkpointing. Then we introduce performance models and use them to assess the performance of theses protocols. We cover the Young/Daly formula for the optimal period and much more! Next we explain how the efficiency of checkpointing can be improved via fault prediction or replication. Then we move to application-specific methods, such as ABFT. We conclude the report by discussing techniques to cope with silent errors (or silent data corruption).This report is a slightly modified version of the first chapter of the monograph Fault tolerance techniques for high-performance computing edited by Thomas Herault and Yves Robert, and to be published by Springer Verlag.

show abstract

Improving cluster availability using workstation validation

Cited by 23 publications

References 0 publications

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Performance Implications of Failures in Large-Scale Cluster Scheduling

Fault-Tolerance Techniques for High-Performance Computing

Contact Info

Product

Resources

About