Fault-Tolerant Protocol for Hybrid Task-Parallel Message-Passing Applications

Martsinkevich, Tatiana V.; Subaşi, Ömer; Ünsal, Osman; Cappello, Franck; Labarta, Jesús

doi:10.1109/cluster.2015.104

Cited by 14 publications

(16 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the area of High-Performance Computing (HPC), parallel systems continue increasing the number of components to improve their performance and, as a consequence, ensuring their reliability has become a critical issue. Nowadays, fault rates involve just a few hours on modern platforms [1] but it is forecasted that large parallel applications will have to manage fault rates of barely some minutes in exascale supercomputers [2]. In that sense, these applications require some help to progress efficiently.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Soft errors detection and automatic recovery based on replication combined with different levels of checkpointing

Montezanti

Rucci

Giusti

et al. 2020

Future Generation Computer Systems

View full text Add to dashboard Cite

Handling faults is a growing concern in HPC. In future exascale systems, it is projected that silent undetected errors will occur several times a day, increasing the occurrence of corrupted results. In this article, we propose SEDAR, which is a methodology that improves system reliability against transient faults when running parallel message-passing applications. Our approach, based on process replication for detection, combined with different levels of checkpointing for automatic recovery, has the goal of helping users of scientific applications to obtain executions with correct results. SEDAR is structured in three levels: (1) only detection and safestop with notification; (2) recovery based on multiple system-level checkpoints; and (3) recovery based on a single valid user-level checkpoint. As each of these variants supplies a particular coverage but involves limitations and implementation costs, SEDAR can be adapted to the needs of the system. In this work, a description of the methodology is presented and the temporal behavior of employing each SEDAR strategy is mathematically described, both in the absence and presence of faults. A model that considers all the fault scenarios on a test application is introduced to show the validity of the detection and recovery mechanisms. An overhead evaluation of each variant is performed with applications involving different communication patterns; this is also used to extract guidelines about when it is beneficial to employ each SEDAR protection level. As a result, we show its efficacy and viability to tolerate transient faults in target HPC environments.

show abstract

Section: Introductionmentioning

confidence: 99%

“…In an unrefined version, our method is an expensive approach, because it needs to keep an undetermined number of active checkpoints and may require several restart attempts. Instead, user-level checkpoints are becoming more usual, especially due to their lower costs and portability options [1].…”

mentioning

confidence: 99%

Soft errors detection and automatic recovery based on replication combined with different levels of checkpointing

Montezanti

Rucci

Giusti

et al. 2020

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…Attempts have been made to add resilience to PaR-SEC [4] and OmpSs [22]. Other work focuses on soft faults [4], i.e., they take advantage of the algorithmic properties of ABFT methods to detect and recover from failures at a fine grain (task level) and utilize periodic Fig.…”

Section: Task-based Resiliencementioning

confidence: 99%

“…Code generated from pragmas with end-to-end resilience for sequentially composed matrix multiplication (mmult is parallelized with OpenMP) checkpointing at a coarse grain (application). Yet others uses CR and message logging at the task granularity to tolerate faults with re-execution [22].…”

Section: Task-based Resiliencementioning

confidence: 99%

DINO: Divergent Node Cloning for Sustained Redundancy in HPC

Rezaei

Mueller

2015

2015 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

A plethora of resilience techniques have been investigated ranging from checkpoint/restart over redundancy to algorithm-based fault tolerance. Each technique works well for a different subset of application kernels, and depending on the kernel, has different overheads, resource requirements, and fault masking capabilities. If, however, such techniques are combined and they interact across kernels, new vulnerability windows are created.This work contributes the idea of end-to-end resilience by protecting windows of vulnerability between kernels guarded by different resilience techniques. It introduces the live vulnerability factor (LVF), a new metric that quantifies any lack of end-to-end protection for a given data structure. The work further promotes end-to-end application protection across kernels via a pragma-based specification, implemented as an extension to OpenMP, for diverse resilience schemes with minimal programming effort. This lifts the data protection burden from application programmers allowing them to focus solely on algorithms and performance while resilience is specified and subsequently embedded into the code through the compiler/library and supported by the runtime system. Two case studies demonstrate that end-to-end resilience meshes well with different execution paradigms and assess its overhead and effectiveness for different codes. In experiments, end-to-end resilience has an overhead over kernel-specific resilience of 1% on average.

show abstract

“…The work of Subasi et al [38] is based on programmer knowledge in order to achieve effective partial replication. The NanoCheckpoints [36] and the message logging protocol proposed by Martsinkevich et al [27] address fail-stop errors of task-parallel computations. SSD [37] is designed by using machine learning techniques to mitigate silent errors in HPC applications.…”

Section: Related Workmentioning

confidence: 99%

A Runtime Heuristic to Selectively Replicate Tasks for Application-Specific Reliability Targets

Subaşi

Yalcin

Zyulkyarov

et al. 2016

2016 IEEE International Conference on Cluster Computing (CLUSTER)

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper we propose a runtime-based selective task replication technique for task-parallel high performance computing applications. Our selective task replication technique is automatic and does not require modification/recompilation of OS, compiler or application code. Our heuristic, we call App FIT, selects tasks to replicate such that the specified reliability target for an application is achieved. In our experimental evaluation, we show that App FIT selective replication heuristic is low-overhead and highly scalable. In addition, results indicate that complete task replication is overkill for achieving reliability targets. We show that with App FIT, we can tolerate pessimistic exascale error rates with only 53% of the tasks being replicated.

show abstract

Fault-Tolerant Protocol for Hybrid Task-Parallel Message-Passing Applications

Cited by 14 publications

References 16 publications

Soft errors detection and automatic recovery based on replication combined with different levels of checkpointing

Soft errors detection and automatic recovery based on replication combined with different levels of checkpointing

DINO: Divergent Node Cloning for Sustained Redundancy in HPC

A Runtime Heuristic to Selectively Replicate Tasks for Application-Specific Reliability Targets

Contact Info

Product

Resources

About