Energy-efficient Checkpointing in High-throughput Cycle-stealing Distributed Systems

Abstract

(2015) 'Energy-ecient checkpointing in high-throughput cycle-stealing distributed systems.', Electronic notes in theoretical computer science., 310. pp. 65-90. Further information on publisher's website: http://dx. The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that: • a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders. Please consult the full DRO policy for further details. Abstract Checkpointing is a fault-tolerance mechanism commonly used in High Throughput Computing (HTC) environments to allow the execution of long-running computational tasks on compute resources subject to hardware or software failures as well as interruptions from resource owners and more important tasks. Until recently many researchers have focused on the performance gains achieved through checkpointing, but now with growing scrutiny of the energy consumption of IT infrastructures it is increasingly important to understand the energy impact of checkpointing within an HTC environment. In this paper we demonstrate through trace-driven simulation of real-world datasets that existing checkpointing strategies are inadequate at maintaining an acceptable level of energy consumption whilst maintaing the performance gains expected with checkpointing. Furthermore, we identify factors important in deciding whether to exploit checkpointing within an HTC environment, and propose novel strategies to curtail the energy consumption of checkpointing approaches whist maintaining the performance benefits.

Energy-Aware Simulation of Workflow Execution in High Throughput Computing Systems

“…For example, advanced reservations could exploit clusters which are closed, hence aid ability to perform back-filling. Checkpointing and migration, which has already been built into our simulation [36], [37], has been shown to be effective in reducing overheads and energy consumption for individual jobs and would seem applicable to workflows. We are currently investigating novel energy-aware workflow scheduling policies for multi-use clusters of dedicated and non-dedicated resources.…”

Trace-Driven Simulation for Energy Consumption in High Throughput Computing Systems

“…In the case where a job is evicted by interactive user this saves both execution time as the jobs need not restart from the beginning but also energy as effort is not expended repeating the previously performed work. However, careful balancing is required in order to determine how often checkpointing should be performed as too many checkpoints will waste time and energy in performing the checkpoints which will not be used -including time and energy required to move checkpoints to a new resource, whilst too few checkpoints will require more work to be repeated -re-doing the work performed between the last checkpoint and the point of eviction [28].…”

“…5) Checkpointing: Checkpointing can save both time and energy by allowing jobs which are evicted to resume from the last checkpoint. However, as the process of checkpointing consumes both time and energy a careful balance is required to minimise energy consumption [28].…”

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

“…We demonstrate, using real-world datasets, that existing checkpointing strategies are inadequate at maintaining an acceptable level of energy consumption whilst retaining the performance gains expected. We identify factors important in determining whether to exploit checkpointing within an HTC environment, and propose novel strategies to curtail the energy consumption of checkpoint approaches while maintaining the performance benefits [58,60,61].…”

“…In the event of an eviction, this allows the job to resume execution from the last available checkpoint, thus aleviating the time and energy associated with re-running execution which would otherwise have been lost. However, as the process of checkpointing consumes both time and energy -copying checkpoint images around the network -a careful balance is required to minimise energy consumption [58,60,61].…”