Adaptive Task Checkpointing and Replication: Toward Efficient Fault-Tolerant Grids

Chtepen, Maria; Claeys, Filip; Dhoedt, Bart; Turck, Filip De; Demeester, Piet; Vanrolleghem, Peter A.

doi:10.1109/tpds.2008.93

Cited by 74 publications

(53 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several authors have suggested techniques that identify tasks on the critical path, and then making scheduling decisions that attempt to ensure the timely execution of these tasks [32,33]. A widely used technique to cope with soft errors is task replication, the challenge being to avoid over-duplicating tasks so CkptAll CkptNone p fail = 0.01 p fail = 0.001 p fail = 0.0001 12 11 11 11 11 11 11 11 31 28 28 28 31 29 28 as to strike a good balance between fast failure-free executions and resilient executions [34]. Two representative practical frameworks are the NARBIT system [35], which recovers from soft errors via task replication and work stealing, and Nanos [36,37], a runtime system that supports the OpenMP programming model.…”

Section: Soft and Silent Errorsmentioning

confidence: 99%

Checkpointing Workflows for Fail-Stop Errors

Han¹,

Canon²,

Casanova³

et al. 2017

2017 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Abstract:We consider the problem of orchestrating the execution of workflow applications structured as Directed Acyclic Graphs (DAGs) on parallel computing platforms that are subject to fail-stop failures. The objective is to minimize expected overall execution time, or makespan. A solution to this problem consists of a schedule of the workflow tasks on the available processors and of a decision of which application data to checkpoint to stable storage, so as to mitigate the impact of processor failures. For general DAGs this problem is hopelessly intractable. In fact, given a solution, computing its expected makespan is still a difficult problem. To address this challenge, we consider a restricted class of graphs, Minimal Series-Parallel Graphs (M-SPGs). It turns out that many real-world workflow applications are naturally structured as M-SPGs. For this class of graphs, we propose a recursive list-scheduling algorithm that exploits the M-SPG structure to assign sub-graphs to individual processors, and uses dynamic programming to decide which tasks in these sub-gaphs should be checkpointed. Furthermore, it is possible to efficiently compute the expected makespan for the solution produced by this algorithm, using a first-order approximation of task weights and existing evaluation algorithms for 2-state probabilistic DAGs. We assess the performance of our algorithm for production workflow configurations, comparing it to (i) an approach in which all application data is checkpointed, which corresponds to the standard way in which most production workflows are executed today; and (ii) an approach in which no application data is checkpointed. Our results demonstrate that our algorithm strikes a good compromise between these two approaches, leading to lower checkpointing overhead than the former and to better resilience to failure than the latter. To the best of our knowledge, this is the first scheduling/checkpointing algorithm for workflow applications with fail-stop failures that considers workflow structures more general than mere linear chains of tasks.Key-words: workflow, checkpoint, fail-stop error, resilience. Stratégies de checkpoint pour les workflows en présence d'erreurs fatalesRésumé : Ce rapport considère l'ordonnancement de workflows (applications structurées en forme de graphes de tâches acycliques, ou DAGs) sur des plates-formes parallèlesà grandé echelle, soumisesà des erreurs fatales. L'objectif est de minimiser l'espérance du temps total d'exécution, ou makespan. Une solutionà ce problème comprend l'allocation ordonnée des tâches aux processeurs, et les décisions de checkpoint: quelles tâches sont suivies d'un checkpoint? Même pour une solution donnée, le calcul du makespan reste difficile. Nous nous restreignonsà une classe de DAGs particuliers, les graphes séries-parallèles minimaux, ou MSPGs. De nombreux workflows issus des applications ont pour graphe un M-SPG. Pour de tels graphes, nous proposons un algorithme qui utilise la structure récursive du M-SPG pour allouer des sous-graphesà chaque pro...

show abstract

Section: Soft and Silent Errorsmentioning

confidence: 99%

Checkpointing Workflows for Fail-Stop Errors

Han¹,

Canon²,

Casanova³

et al. 2017

2017 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

show abstract

“…It computes a remapping interval during which it remaps those jobs that are assigned to a faulty resource and are inactive to some other resource in advance before it begins its execution. A minmax checkpoint placement method [1] is introduced that determines the suboptimal checkpoint sequence under uncertain circumstances in terms of the system failure time distribution. However, even if the (sub)optimal checkpointing interval is computed beforehand, the distributed system or application parameters upon which the interval is based will presumably change over time.…”

Section: Related Workmentioning

confidence: 99%

“…The grid model (Figure 1) considered in this paper consists of [1]: geographically distributed computational sites with many computational resources (r) at each site. The latter include a user interface (UI) through which the jobs are submitted into the system; a Resource Broker (RB) which is used to identify all the available resources, a scheduler(S) to schedule the job to the available resources.…”

Section: Proposed Workmentioning

confidence: 99%

“…The main aim of this Last Failure time based Checkpoint Adaptation (LFCA) algorithm [1] is to omit unnecessary checkpoint in-order to reduce the checkpoint overhead on a relatively stable resource. This unnecessary checkpoints are omitted mainly to reduce the overhead and to increase the job throughput.…”

Section: Last Failure Time Based Checkpoint Adaptationmentioning

confidence: 99%

“…The Mean Failure based Checkpoint Adaptation (MFCA) algorithm [1] dynamically modifies the checkpointing frequency and deal with inappropriate checkpointing intervals. The checkpointing frequency is modified based on the Remaining job execution time (RE j r ) and mean failure interval of the resource (Mf r ) where r is the resource and j is the job assigned to that resource.…”

Section: Mean Failure Time Based Checkpoint Adaptationmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Adaptation of Checkpoints and Rescheduling in Grid Computing

Therasa.¹,

Sumathi²,

Dalya.³

2010

IJCA

View full text Add to dashboard Cite

Grid is a form distributed computing mainly to virtualilze and utilize geographically distributed idle resources. A grid is a distributed computational and storage environment often composed of heterogeneous autonomously managed subsystems. As a result varying resource availability becomes common place, often resulting in loss and delay of executing jobs. To ensure good performance fault tolerance should be taken into account. Here we address the fault tolerance in terms of resource failure. Commonly utilized techniques to achieve fault tolerance is periodic checkpointing, which periodically saves the jobs state. But an inappropriate checkpointing interval leads to delay in the job execution, and reduces the throughput. Hence in the proposed work, the strategy used to achieve fault tolerance is by dynamically adapting the checkpoints based on current status and history of failure information of the resource, which is maintained in the Information server. The Last failure time and Mean failure time based algorithm dynamically modifies the frequency of checkpoint interval, hence increases the throughput by reducing the unnecessary checkpoint overhead. In case of resource failure, the proposed Fault Index Based Rescheduling (FIBR) algorithm reschedules the job from the failed resource to some other available resource with the least Fault-index value and executes the job from the last saved checkpoint. This ensures the job to be executed within the deadline with increased throughput and helps in making the grid environment trust worthy.

show abstract

Failover strategy for fault tolerance in cloud computing environment

et al. 2017

View full text Add to dashboard Cite

Summary Cloud fault tolerance is an important issue in cloud computing platforms and applications. In the event of an unexpected system failure or malfunction, a robust fault‐tolerant design may allow the cloud to continue functioning correctly possibly at a reduced level instead of failing completely. To ensure high availability of critical cloud services, the application execution, and hardware performance, various fault‐tolerant techniques exist for building self‐autonomous cloud systems. In comparison with current approaches, this paper proposes a more robust and reliable architecture using optimal checkpointing strategy to ensure high system availability and reduced system task service finish time. Using pass rates and virtualized mechanisms, the proposed smart failover strategy (SFS) scheme uses components such as cloud fault manager, cloud controller, cloud load balancer, and a selection mechanism, providing fault tolerance via redundancy, optimized selection, and checkpointing. In our approach, the cloud fault manager repairs faults generated before the task time deadline is reached, blocking unrecoverable faulty nodes as well as their virtual nodes. This scheme is also able to remove temporary software faults from recoverable faulty nodes, thereby making them available for future request. We argue that the proposed SFS algorithm makes the system highly fault tolerant by considering forward and backward recovery using diverse software tools. Compared with existing approaches, preliminary experiment of the SFS algorithm indicates an increase in pass rates and a consequent decrease in failure rates, showing an overall good performance in task allocations. We present these results using experimental validation tools with comparison with other techniques, laying a foundation for a fully fault‐tolerant infrastructure as a service cloud environment. Copyright © 2017 John Wiley & Sons, Ltd.

show abstract

Adaptive Task Checkpointing and Replication: Toward Efficient Fault-Tolerant Grids

Cited by 74 publications

References 26 publications

Checkpointing Workflows for Fail-Stop Errors

Checkpointing Workflows for Fail-Stop Errors

Dynamic Adaptation of Checkpoints and Rescheduling in Grid Computing

Failover strategy for fault tolerance in cloud computing environment

Contact Info

Product

Resources

About