Flipping the priority: effects of prioritising HTC jobs on energy consumption in a multi-use cluster

Abstract

Further information on publisher's website:http://dx.doi.org/10.4108/eai. 24-8-2015.2261101 Publisher's copyright statement:Additional information: Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. ABSTRACTHigh Throughput Computing (HTC) through the use of volunteer computing provides a compelling opportunity to perform large volumes of computation without the need to invest in computational resources. This relies on the good will of computer owners who volunteer their computer idle time. In scenarios when there is contention the HTC system will relinquish the computer -normally achieved through termination or suspension. This often leads to longer turn-around times for the HTC jobs and an increase in energy consumption when the work is restarted elsewhere. However, within large organisations this clear distinction of who is the owner of a computer and who should take priority is not always clear. If a user enters a large cluster room (of identical computers) should they be allowed to use a computer which is servicing HTC jobs when other computers are idle? Should HTC jobs be able to delay the rebooting of computers thus allowing the jobs to complete? In this work we relax some of the common policies used in management of computers in large organisations in order to evaluate if alternative policies which may have impact on the primary users of the computers could save enough energy to make this impact tolerable. We evaluate our approach through the use of the HTC-Sim simulation framework. We demonstrate a potential energy saving of 12.4%, along with overhead reductions of 20-74%, and up to 48% reduction in job terminations by re-directing only 13% of users away from computers servicing HTC jobs.

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

“…However, such systems are typically configured to prioritise the interactive user of the computer in situations where there is resource contention between users and HTC jobs. In [72] we relax some of the common policies used in management of computers in large organisations in order to evaluate if alternative policies which may have a minor impact on the primary users of the computers could save enough energy to make this impact tolerable. We focus on our scenario of institutional clusters, augmenting reboot policies to reduce their impact on HTC workloads, and introducing a scheme whereby student users arriving to a cluster are directed away from computers currently servicing HTC jobs.…”