An energy management system for cluster infrastructures

Abstract

This paper presents a general energy management system for High Performance Computing (HPC) clusters and cloud infrastructures that powers off cluster nodes when they are not being used, and conversely powers them on when they are needed. This system can be integrated with different HPC cluster middleware, such as Batch-Queuing Systems or Cloud Management Systems, and can also use different mechanisms for powering on and off the computing nodes. The presented system makes it possible to implement different energy-saving policies depending on the priorities and particularities of the cluster. It also provides a hook system to extend the functionality, and a sensor system in order to take into account environmental information. The paper describes the successful integration of the system proposed with some popular Batch-Queuing Systems, and also with some Cloud Management middlewares, presenting two real use-cases that show significant energy/costs savings of 27% and 17%.

Efficient and elastic management of computing infrastructures

“…Based on this information, the scheduler determines if new nodes must be switched on, or if there are nodes that can be switched off, and acts consequently (2). When a job is submitted to 22 the resource manager (3), a request for nodes is made to CLUES by means of the resource manager connector (4). When CLUES finishes processing this request, the job is actually submitted to the resource manager (5), where it will be processed by the scheduler and finally sent to the cluster for execution (6).…”

“…For simplicity, the use case does not consider using reserved instances to decrease the cost of the allocated resources in the Cloud. However, as shown in [4], a proper selection of the number of reserved instances, considering the cluster workload, can deliver significant cost savings. In [6], the power consumption of the clusters was measured to get the energy consumed and its associated cost.…”

“…So the key objective for the analysis of VMCA is to move every VM from a physical host to the other servers that are already hosting a VM to get some physical nodes idle. The final consequence is that the idle nodes may be suspended or powered off by using other software such as CLUES [4].…”

“…Amazon itself actually built a supercomputer on its own Cloud that made it onto the list of the world's Top 500 supercomputers. With 7,000 cores, that specific Amazon cluster ranked number 232 in the world in November 2010 with speeds of 41.82 teraflops, falling to number 451 in June of this year 4 . It is estimated that the whole Amazon EC2 infrastructure can be ranked number 42 among the world's Top 500 supercomputers 5 .…”

“…In a previous work [4] we concluded that, in some cases, it is interesting to deploy a virtual cluster instead of a physical one. A virtual cluster in an IaaS provider is able to get a competitive performance per price rate, but also gets important benefits from the Cloud provider such as reducing the administration costs (both personnel costs and maintenance of equipments), avoiding hiring or buying the physical building to host the infrastructure, avoiding the upfront investments in hardware, cooling systems, etc.…”