A Large-Scale Study of Failures in High-Performance Computing Systems

Schroeder, Bianca; Gibson, Garth A.

doi:10.1109/tdsc.2009.4

Cited by 492 publications

(232 citation statements)

References 16 publications

Supporting

Mentioning

231

Contrasting

Order By: Relevance

“…The failure number actually grows over a period of nearly 18 months, before it eventually starts dropping. The reason most likely is that many problems in hardware, software and configuration are only exposed by real user code in the production workloads [3].…”

Section: ) System Failuresmentioning

confidence: 99%

Study and Analysis of the High Performance Computing Failures in China Meteorological Field

Chen¹,

Sun²

2017

GEP

View full text Add to dashboard Cite

China Meteorological Administration (CMA) has a long history of using High Performance Computing System (HPCS) for over three decades. CMA HPCS investment provides reliable HPC capabilities essential to run Numerical Weather Prediction (NWP) models and climate models, generating millions of weather guidance products daily and providing support for Coupled Model Inter-comparison Project Phase 5 (CMIP5). Monitoring the HPCS and analyzing the resource usage can improve the performance and reliability for our users, which require a good understanding of failure characteristics. Large-scale studies of failures in real production systems are scarce. This paper collects, analyzes and studies all the failures occurring during the HPC operation period, especially focusing on studying the relationship between HPCS and NWP applications. Also, we present the challenges for a more effective monitoring system development and summarize the useful maintenance strategies. This step may have considerable effects on the performance of online failure prediction of HPC and better performance in future.

show abstract

Section: ) System Failuresmentioning

confidence: 99%

Study and Analysis of the High Performance Computing Failures in China Meteorological Field

Chen¹,

Sun²

2017

GEP

View full text Add to dashboard Cite

show abstract

“…For each physical node, the MTBF is programmed according to a Weibull distribution, with a shape parameter of 0.8, which has been shown [37] to well approximate the time between failures for individual nodes, as well as for the entire system. Failed nodes stay unavailable (i.e., mean time to repair (MTTR)) during a period modelled by a lognormal distribution, with a mean time set to 20 min, varying up to 150 min.…”

Section: Failures and Unavailability Propertiesmentioning

confidence: 99%

Towards high-available and energy-efficient virtual computing environments in the cloud

Sampaio

Barbosa

2014

Future Generation Computer Systems

View full text Add to dashboard Cite

h i g h l i g h t s• A review of failure-aware algorithms for resource provisioning on clouds.• The development of power-and failure-aware cloud scheduling algorithms that implement vertical and horizontal platform elasticity.• Autonomous readjustment of virtual-to-physical mappings on-the-fly.• Extensive evaluation of the dynamic strategy and of the scheduling algorithms.• Random workloads and workloads that follow the Google cloud tracelogs. a r t i c l e i n f o t r a c tEmpowered by virtualisation technology, cloud infrastructures enable the construction of flexible and elastic computing environments, providing an opportunity for energy and resource cost optimisation while enhancing system availability and achieving high performance. A crucial requirement for effective consolidation is the ability to efficiently utilise system resources for high-availability computing and energy-efficiency optimisation to reduce operational costs and carbon footprints in the environment. Additionally, failures in highly networked computing systems can negatively impact system performance substantially, prohibiting the system from achieving its initial objectives. In this paper, we propose algorithms to dynamically construct and readjust virtual clusters to enable the execution of users' jobs. Allied with an energy optimising mechanism to detect and mitigate energy inefficiencies, our decisionmaking algorithms leverage virtualisation tools to provide proactive fault-tolerance and energy-efficiency to virtual clusters. We conducted simulations by injecting random synthetic jobs and jobs using the latest version of the Google cloud tracelogs. The results indicate that our strategy improves the work per Joule ratio by approximately 12.9% and the working efficiency by almost 15.9% compared with other state-ofthe-art algorithms.

show abstract

“…Noticeable progress has been made on failure prediction research and practice, following that more failure traces are made public available since 2006 and the failure analysis [5][6][7][8][9][10] [11] develop a resource failure prediction model for fine-grained cycle sharing systems. However, most of them focus on improving the predication accuracy, and few of them address how to leverage their predication results in practice.…”

Section: Related Workmentioning

confidence: 99%

“…Studies in [5][6][7][8][9] recognize the temporal and spatial correlation of failures in large scale cluster systems. In order to mimic these failure characteristics in the simulations, we choose to use the failure traces published by Los Alamos National Laboratory [23].…”

Section: Failure Traces and Prediction Accuracymentioning

confidence: 99%

See 1 more Smart Citation

Failure-aware workflow scheduling in cluster environments

Wang

Shi

2010

Cluster Comput

View full text Add to dashboard Cite

The goal of workflow application scheduling is to achieve minimal makespan for each workflow. Scheduling workflow applications in high performance cluster environments is an NP-Complete problem, and becomes more complicated when potential resource failures are considered. While more research on failure prediction has been witnessed in recent years to improve system availability and reliability, very few of them attack the problem in the context of workflow application scheduling. In this paper, we study how a workflow scheduler benefits from failure prediction and propose FLAW, a failure-aware workflow scheduling algorithm. We propose two important definitions on accuracy, Application Oblivious Accuracy (AOA) and Application Aware Accuracy (AAA), from the perspectives of system and scheduling respectively, as we observe that the prediction accuracy defined conventionally imposes different performance implications on different applications and fails to measure how that improves scheduling effectiveness. The comprehensive evaluation results using real failure traces show that FLAW performs well with practically achievable prediction accuracy by reducing the average makespan, the loss time and the number of job rescheduling.

show abstract

A Large-Scale Study of Failures in High-Performance Computing Systems

Cited by 492 publications

References 16 publications

Study and Analysis of the High Performance Computing Failures in China Meteorological Field

Study and Analysis of the High Performance Computing Failures in China Meteorological Field

Towards high-available and energy-efficient virtual computing environments in the cloud

Failure-aware workflow scheduling in cluster environments

Contact Info

Product

Resources

About