Online Server and Workload Management for Joint Optimization of Electricity Cost and Carbon Footprint Across Data Centers

Abbasi, Zahra; Pore, Madhurima; Gupta, Sandeep

doi:10.1109/ipdps.2014.42

Cited by 14 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In doing so, they designed a heuristic smoothing algorithm and an efficient distributed randomized online algorithm. Abbasi et al [34] proposed an online algorithm to minimize operational cost of a set of geo-distributed data centers. Song et al [35] proposed an online bin packing approach that uses virtualization technology to allocate cloud resources dynamically based on application demands.…”

Section: Related Workmentioning

confidence: 99%

An Online Mechanism for Resource Allocation and Pricing in Clouds

Mashayekhy

Nejad

Grosu

et al. 2016

IEEE Trans. Comput.

136

View full text Add to dashboard Cite

Cloud providers provision their various resources such as CPUs, memory, and storage in the form of Virtual Machine (VM) instances which are then allocated to the users. The users are charged based on a pay-as-you-go model, and their payments should be determined by considering both their incentives and the incentives of the cloud providers. Auction markets can capture such incentives, where users name their own prices for their requested VMs. We design an auction-based online mechanism for VM provisioning, allocation, and pricing in clouds that consider several types of resources. Our proposed online mechanism makes no assumptions about future demand of VMs, which is the case in real cloud settings. The proposed online mechanism is invoked as soon as a user places a request or some of the allocated resources are released and become available. The mechanism allocates VM instances to selected users for the period they are requested for, and ensures that the users will continue using their VM instances for the entire requested period. In addition, the mechanism determines the payment the users have to pay for using the allocated resources. We prove that the mechanism is incentive-compatible, that is, it gives incentives to the users to reveal their actual requests. We investigate the performance of our proposed mechanism through extensive experiments.

show abstract

Section: Related Workmentioning

confidence: 99%

An Online Mechanism for Resource Allocation and Pricing in Clouds

Mashayekhy

Nejad

Grosu

et al. 2016

IEEE Trans. Comput.

136

View full text Add to dashboard Cite

show abstract

“…• Randomness is incorporated in E-Ant by the fact that its task assignment policy is based on probability as shown in Equation 3. This allows E-Ant to explore new solutions which may lead to a globally optimal solution, instead of getting stuck in a local optimum.…”

Section: B Energy Consumption Modelmentioning

confidence: 99%

“…Hence, S pool is equal to the total number of slots available in the cluster. E-Ant incorporates the above heuristic information into its task assignment policy by updating Equation 3. The probability that E-Ant will assign a task of the j th job to the m th machine is now given by:…”

Section: ) Heuristic Informationmentioning

confidence: 99%

“…This trend is driving up the need for designing energy-efficient Hadoop clusters in order to reduce the operational costs and the carbon emission associated with its energy consumption [3], [20], [22]. However prevalent power management techniques, which involve server consolidation and turning off idle machines, are not readily applicable to Hadoop clusters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Energy Efficiency in Heterogeneous Hadoop Clusters by Adaptive Task Assignment

Cheng

Lama

Jiang

et al. 2015

2015 IEEE 35th International Conference on Distributed Computing Systems

View full text Add to dashboard Cite

The cost of powering servers, storage platforms and related cooling systems has become a major component of the operational costs in big data deployments. Hence, the design of energy-efficient Hadoop clusters has attracted significant research attention in recent years. However, existing studies do not consider the impact of the complex interplay between workload and hardware heterogeneity on energy efficiency. In this paper, we find that heterogeneity-oblivious task assignment approaches are detrimental to both performance and energy efficiency of Hadoop clusters. Importantly, we make a counterintuitive observation that even heterogeneity-aware techniques that focus on reducing job completion time do not necessarily guarantee energy efficiency. We propose a heterogeneity-aware task assignment approach, E-Ant, that aims to minimize the overall energy consumption in a heterogeneous Hadoop cluster without sacrificing job performance. It adaptively schedules heterogeneous workloads on energy-efficient machines, without a priori knowledge of the workload properties. Furthermore, it provides the flexibility to tradeoff job fairness and energy efficiency in a Hadoop cluster. E-Ant employs an ant colony optimization approach to generate task assignment solutions based on the feedback of each task's energy consumption reported by Hadoop TaskTrackers. Experimental results on a heterogeneous cluster with varying hardware capabilities show that E-Ant improves the overall energy savings for a synthetic workload from Microsoft by 17% and 12% compared to Fair Scheduler and Tarazu respectively.

show abstract

“…Internet has witnessed a dramatic increase in the popularity of cloud computing systems, which rent resources on demand and bill on a pay-as-you-go basis. There is an extensive literature on cloud-based resource scheduling for multiple services [14][15][16][17][18][19]. Lee [16] solves the cloud resource management problem from providers' perspective and users' viewpoint.…”

Section: Related Workmentioning

confidence: 99%

FRP: a fast resource placement algorithm in distributed cloud computing platform

Wei

Zhang

Liu

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

Summary We consider a large‐scale online service system of placing resources geographically distributed over multiple regional cloud data centers. Service providers need to place the resources in these regions so as to maximize profit, accounting for demand granting revenues minus resource placement costs. The challenge is how to optimally place these resources to fulfill varying demands (e.g., multidimensional and stochastic demands) among these cloud data centers. Considering demand stochasticity will significantly increase time complexity of resource placement algorithm, resulting in inefficiency when handling a large number of resources. We propose a fast resource placement algorithm (FRP) to obtain the maximum resource revenue from distributed cloud systems. Experiments show that in scenarios with general settings, FRP can achieve up to 99.2% revenue of existed best solution while reducing execution time by two orders of magnitude. Therefore, FRP is an effective supplement to existing algorithms under time‐tense scheduling scenarios with a large number of resources. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

Online Server and Workload Management for Joint Optimization of Electricity Cost and Carbon Footprint Across Data Centers

Cited by 14 publications

References 14 publications

An Online Mechanism for Resource Allocation and Pricing in Clouds

An Online Mechanism for Resource Allocation and Pricing in Clouds

Towards Energy Efficiency in Heterogeneous Hadoop Clusters by Adaptive Task Assignment

FRP: a fast resource placement algorithm in distributed cloud computing platform

Contact Info

Product

Resources

About