Energy Efficient Scheduling of MapReduce Jobs

Bampis, Evripidis; Chau, Vincent; Letsios, Dimitrios; Lucarelli, Giorgio; Milis, Ioannis; Zois, Georgios

doi:10.1007/978-3-319-09873-9_17

Cited by 18 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notice that map-reduce is a standard programming model widely used in data centers, where the energy consumption is one of the most important issues which has to be handled by the companies and the designers. In this direction, the map-reduce/concurrent open-shop problem in combination with the speed-scaling mechanism have been studied in [8], with objective to minimize the sum of the weighted completion times jobs under a given budget of energy. Finally, note that the results we present in this section hold also for the energy minimization concurrent open-shop problem.…”

Section: Energy Minimization On An Open-shop With Preemptionsmentioning

confidence: 99%

Green scheduling, flows and matchings

Bampis

Letsios

Lucarelli

2015

Theoretical Computer Science

Self Cite

View full text Add to dashboard Cite

International audienceRecently, optimal combinatorial algorithms have been presented for the energy minimization multiprocessor speed-scaling problem with migrations [5] and [7]. These algorithms use repeated maximum-flow computations that allow the partition of the set of jobs into subsets in which all the jobs are executed at the same speed. The optimality of these algorithms is based on a series of technical lemmas showing that this partition and the corresponding speeds lead to the minimization of the energy consumption. In this paper, we show that both the algorithms and their analysis can be greatly simplified. In order to do this, we formulate the problem as a convex cost flow problem in an appropriate flow network. Furthermore, we show that our approach is useful to solve other problems in the dynamic speed-scaling setting. As an example, we consider the preemptive open-shop speed-scaling problem and we propose a polynomial-time algorithm for finding an optimal solution based on the computation of convex cost flows. We also propose a polynomial-time algorithm for minimizing a linear combination of the sum of the completion times of the jobs and the total energy consumption, for the non-preemptive multiprocessor speed-scaling problem. Instead of using convex cost flows, our algorithm is based on the computation of a minimum weighted maximum matching in an appropriate bipartite graph

show abstract

Section: Energy Minimization On An Open-shop With Preemptionsmentioning

confidence: 99%

Green scheduling, flows and matchings

Bampis

Letsios

Lucarelli

2015

Theoretical Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…[40] Energy Efficient Scheduling of MapReduce Jobs Considers minimizing the total weighted completion time of jobs under a given budget of energy.…”

Section: Energy Efficient Mapreduce Techniquesmentioning

confidence: 99%

Analysis of MapReduce scheduling and its improvements in cloud environment

D’Souza

Chandrasekaran

2015

2015 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES)

View full text Add to dashboard Cite

MapReduce has become a prominent Parallel processing model used for analysing large scale data. MapReduce applications are increasingly being deployed in the cloud along with other applications sharing the same physical resources. In this scenario, efficient scheduling of MapReduce applications is of utmost importance. Also, MapReduce has to consider various other parameters like energy efficiency and meeting SLA goals besides achieving performance when executing jobs in cloud environments. In this work, we have classified MapReduce Scheduling as Cluster based Scheduling and Objective based Scheduling. We then summarize and analyse the different class of schedulers highlighting the strong points and limitations of each of the scheduling approaches. The Adaptive scheduling techniques provide dynamic resource management and meet performance goals. The Energy efficient scheduling techniques aim to cut data centre costs by using different approaches. Finally, we discuss the current challenges and future work.

show abstract

“…In prior work, there is a large body of work on online algorithms for the flow time plus energy problem, however, to the best of our knowledge except [27], [3], does not consider any precedence constraints. Without precedence constraints, online algorithms for multiple servers to minimize flow time and energy with constant competitive ratios have been derived in [19], [13], [4], [2], [16], [20], [15], [1], [10], under both the homogenous server model, i.e., P (.)…”

Section: Introductionmentioning

confidence: 99%

“…The most relevant work for our considered problem is [27], [24], [3], that considers multiple servers with tuneable speed, and the objective is to find an optimal algorithm that minimizes the completion time under a total energy constraint for arbitrary precedence constraints defined by a directed acyclic graph (DAG) [27] and for MapReduce type constraints [24], [3]. However, the results of [27] are limited for the case when all jobs are available at time 0, and its approximation ratio scales as square of the logarithm of the number of servers.…”

Section: Introductionmentioning

confidence: 99%

“…However, the results of [27] are limited for the case when all jobs are available at time 0, and its approximation ratio scales as square of the logarithm of the number of servers. In [24], [3], an offline problem (where the exact arrival sequence of jobs is known a priori) to minimize the completion time with MapReduce precedence constraints is considered. Moreover, [3] requires that for each task there is a preassigned server that only is allowed to process it.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Speed Scaling On Parallel Servers with MapReduce Type Precedence Constraints

Vaze¹,

Nair²

2021

Preprint

View full text Add to dashboard Cite

A multiple server setting is considered, where each server has tunable speed, and increasing the speed incurs an energy cost. Jobs arrive to a single queue, and each job has two types of sub-tasks, map and reduce, and a precedence constraint among them: any reduce task of a job can only be processed once all the map tasks of the job have been completed. In addition to the scheduling problem, i.e., which task to execute on which server, with tunable speed, an additional decision variable is the choice of speed for each server, so as to minimize a linear combination of the sum of the flow times of jobs/tasks and the total energy cost. The precedence constraints present new challenges for the speed scaling problem with multiple servers, namely that the number of tasks that can be executed at any time may be small but the total number of outstanding tasks might be quite large. We present simple speed scaling algorithms that are shown to have competitive ratios, that depend on the power cost function, and/or the ratio of the size of the largest task and the shortest reduce task, but not on the number of jobs, or the number of servers.

show abstract

Energy Efficient Scheduling of MapReduce Jobs

Cited by 18 publications

References 17 publications

Green scheduling, flows and matchings

Green scheduling, flows and matchings

Analysis of MapReduce scheduling and its improvements in cloud environment

Speed Scaling On Parallel Servers with MapReduce Type Precedence Constraints

Contact Info

Product

Resources

About