Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters

Niu, Shuangcheng; Zhai, Jidong; Ma, Xiaosong; Tang, Xiongchao; Chen, Wenguang

doi:10.1145/2503210.2503236

Cited by 39 publications

(26 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aiming at decreasing the cost of executing highperformance computing (HPC) applications in cloud environments, Niu and Tang [7] proposed a Semi-Elastic Cluster (SEC) computing model to reserve and dynamically resize a virtual cloud-based cluster. The idea is to aggregate the workloads from concurrent users and enjoy from deep discount to heavy users through reserved instances, a policy that has been done by some cloud resource providers such as Amazon.…”

Section: Related Workmentioning

confidence: 99%

“…Since task scheduling is a well-known NPcomplete problem [4], several heuristic algorithms have been proposed for the scheduling of workflows onto distributed systems like Grids and Clouds [5], [6], [7]. These scheduling algorithms follow some cost model, in order to optimize specific objectives, such as execution time, economic cost, and data quality.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Study on Cloud Cost Efficiency by Exploiting Idle Billing Period Fractions

Sampaio

Barbosa

2016

2016 IEEE International Conference on Cloud Engineering Workshop (IC2EW)

View full text Add to dashboard Cite

Abstract-In most of the current commercial Clouds, resources are billed based on a time interval equal to one hour, as is the case of virtual machine (VM) instances on Amazon EC2. Such time interval is usually long, and yet the user has to pay for the whole last hour, even if he/she has only used a fraction of it, contradicting the pay-as-you-go model of Clouds.In this paper, we analyse the advantages of adopting alternative scheduling policies that exploit idle last time intervals, in terms of service cost to Cloud users and operating costs to Cloud providers. Using a real-life astronomy workflow application, constrained by user-defined Deadline and Budget quality of service (QoS) parameters, a set of online state-ofthe-art-based scheduling algorithms try different execution and resource provisioning plans. Our results show that exploitation of partially idle last time intervals can reduce the cost of service to the end user, and augments providers competitiveness up to 21.6% through energy efficiency improvement and consequent lowering of operational costs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Study on Cloud Cost Efficiency by Exploiting Idle Billing Period Fractions

Sampaio

Barbosa

2016

2016 IEEE International Conference on Cloud Engineering Workshop (IC2EW)

View full text Add to dashboard Cite

show abstract

“…Other tools to deploy virtual clusters can be found in the literature, such as Wrangler [3] or the work by Niu et al [4]. The former does not support elasticity while the latter, although it does include elasticity rules to scale the clusters, it does not consider support for spot instances, which is a cost-effective mechanism to provision computational resources for interruptible tasks, supported by Amazon EC2.…”

Section: Introductionmentioning

confidence: 99%

Towards Migratable Elastic Virtual Clusters on Hybrid Clouds

Calatrava

Moltó

Romero

et al. 2015

2015 IEEE 8th International Conference on Cloud Computing

View full text Add to dashboard Cite

Abstract-This paper describes the research work in the context of the CLUVIEM project towards achieving migratable, self-managed virtual elastic clusters on hybrid Cloud infrastructures. These virtual clusters can span across onpremises and public Cloud infrastructures thus leveraging hybrid Cloud platforms. They are elastic since working nodes are automatically provisioned and relinquished to dynamically adapt the capacity of the virtual cluster (in terms of number of nodes) according to the current workload. They are selfmanaged since the elasticity rules are managed via the head node without requiring any external software entity for monitoring and deciding when to scale in and out. Finally, they are migratable since they consider both application migration, via application checkpointing, and infrastructure migration, by cloning infrastructures across multi-Clouds. These features introduce unprecedented flexibility for cost-effective clusterbased computing with minimal impact for cluster users. The paper summarises the current state of developments and future roads to achieve this vision.

show abstract

“…As an example, [15] and [84] that evaluate the possibility of using Amazon EC2 to extend a physical cluster, depending on the workload. The work [99] also explores the dynamic provision of working nodes in the cloud, depending on the size of the jobs in the queues, introducing several policies to limit the amount of working nodes to be powered on. The main limitation of most works in this field is that they seem to be ad-hoc private implementations that have not been released or they remain as theoretical works.…”

Section: Elastic Virtual Clustersmentioning

confidence: 99%

Efficient and elastic management of computing infrastructures

Laguna¹,

de²

View full text Add to dashboard Cite

AcknowledgementsEn primer lugar quiero dar las gracias a mis padres Lola y Ramón, por haberme animado siempre a continuar trabajando. Gracias a ellos he ido avanzando en mi vida y he llegado hasta aquí. Están tan orgullosos, que sólo por oírles hablar de mí, merece la pena seguir haciendo cosas. Muchas gracias papás, espero seguir consiguiendo que os sintáis orgullosos de mí.Quiero dar las gracias a Mar porque siempre está a mi lado y por haberme apoyado tanto para terminar este trabajo. Ha estado todo el tiempo ahí animándome, diciéndome lo poco que me iba a costar terminarlo, y que no me preocupara, porque iba a quedar bien. Muchas gracias mi amor, espero que sigas siempre junto a mí.También tengo que agradecer la labor de Miguel Caballer y Fernando Alvarruíz, porque con ellos empezó esta tesis y porque han aportado mucho trabajo a ella. Y agradecer a Nacho Blanquer y a Germán Moltó, que se hayan ido incorporando al desarrollo del trabajo, y se hayan implicado de tal forma que han acabado siendo mis directores. Muchas gracias a todos, espero poder continuar trabajando con vosotros.Y finalmente, quiero dar las gracias a Vicente Hernández, porqueél es el artífice de que hace años empezase con esto de la investigación y sé que, aunque no me haya podido acompañar hasta aquí, se habría sentido orgulloso al ver que por fin he terminado. Muchas gracias, Vicente.iii AbstractModern data centers integrate a lot of computer and electronic devices. However, some reports state that the mean usage of a typical data center is around 50% of its peak capacity, and the mean usage of each server is between 10% and 50%. A lot of energy is destined to power on computer hardware that most of the time remains idle. Therefore, it would be possible to save energy simply by powering off those parts from the data center that are not actually used, and powering them on again as they are needed.Most data centers have computing clusters that are used for intensive computing, recently evolving towards an on-premises Cloud service model. Despite the use of low consuming components, higher energy savings can be achieved by dynamically adapting the system to the actual workload. The main approach in this case is the usage of energy saving criteria for scheduling the jobs or the virtual machines into the working nodes. The aim is to power off idle servers automatically. But it is necessary to schedule the power management of the servers in order to minimize the impact on the end users and their applications.The objective of this thesis is the elastic and efficient management of cluster infrastructures, with the aim of reducing the costs associated to idle components. This objective is addressed by automating the power management of the working nodes in a computing cluster, and also proactive stimulating the load distribution to achieve idle resources that could be powered off by means of memory overcommitment and live migration of virtual machines. Moreover, this automation is of interest for virtual clusters, as they also suffer from the same ...

show abstract

Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters

Cited by 39 publications

References 28 publications

A Study on Cloud Cost Efficiency by Exploiting Idle Billing Period Fractions

A Study on Cloud Cost Efficiency by Exploiting Idle Billing Period Fractions

Towards Migratable Elastic Virtual Clusters on Hybrid Clouds

Efficient and elastic management of computing infrastructures

Contact Info

Product

Resources

About