Optimal online algorithms for MapReduce scheduling on two uniform machines

Jiang, Yiwei; Ping, Zhou; Cheng, T.C.E.; Ji, Min

doi:10.1007/s11590-018-01384-8

Cited by 7 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large number of studies [18][19][20][21][22][23][27][28][29][30] have been conducted to minimize the makespan of jobs and improve Hadoop performance. We classified the works into two categories: (i) Studies ignoring resource and workload heterogeneity, (ii) Studies considering the heterogeneity in terms of resource and workload.…”

Section: Related Workmentioning

confidence: 99%

“…The preemption of jobs has been taken into account to achieve fairness of jobs. In [20], Jiang et al have presented an online scheduler with the objective of minimization of makespan of MapReduce jobs. Authors have considered both preemptive and nonpreemptive Reduce tasks in a homogeneous Hadoop cluster.…”

Section: Related Workmentioning

confidence: 99%

“…The studies that aim at improving the parallel performance of MapReduce job either try to schedule only the Reduce tasks to diminish data transmission cost in the shuffle phase [7,[15][16][17] or try to minimize the job completion time by only considering scheduling Map or Reduce tasks [18][19][20][21][22][23] (see "Related work" section). They usually focus on only the assignment of Reduce tasks with the assumption that Map scheduling is determined by the initial data distribution of the file system hosted on the MapReduce compute nodes.…”

mentioning

confidence: 99%

See 2 more Smart Citations

TMaR: a two-stage MapReduce scheduler for heterogeneous environments

Maleki

Faragardi

Rahmani

et al. 2020

Hum. Cent. Comput. Inf. Sci.

View full text Add to dashboard Cite

In the context of MapReduce task scheduling, many algorithms mainly focus on the scheduling of Reduce tasks with the assumption that scheduling of Map tasks is already done. However, in the cloud deployments of MapReduce, the input data is located on remote storage which indicates the importance of the scheduling of Map tasks as well. In this paper, we propose a two-stage Map and Reduce task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce tasks on the servers that minimize the task finish time in each stage, respectively. We employ a dynamic partition binder for Reduce tasks in the Reduce stage to lighten the shuffling traffic. Indeed, TMaR minimizes the makespan of a batch of tasks in heterogeneous environments while considering the network traffic. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network traffic and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

TMaR: a two-stage MapReduce scheduler for heterogeneous environments

Maleki

Faragardi

Rahmani

et al. 2020

Hum. Cent. Comput. Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…From the small-scale instances, it can be seen that when the range of the processing time of operations is enlarged from (1,5) to (1,10), the running time of CPLEX increases significantly. While the running time of the genetic algorithm, the simulated annealing algorithm, and L-F algorithm does not change significantly.…”

Section: Small-scale Instancesmentioning

confidence: 99%

“…Jiang et al [9] propose an online algorithm which can optimize their MapReduce problem, where the map operations are fractional and the reduce operations are preemptive. e paper [10] studies an online MapReduce scheduling problem on two uniform machines and discusses the competitive ratio with preemption or nonpreemption, respectively.…”

Section: Introductionmentioning

confidence: 99%

Heuristic Algorithms for MapReduce Scheduling Problem with Open-Map Task and Series-Reduce Tasks

Zheng

Wang

et al. 2020

Scientific Programming

View full text Add to dashboard Cite

Based on the classical MapReduce concept, we propose an extended MapReduce scheduling model. In the extended MapReduce scheduling problem, we assumed that each job contains an open-map task (the map task can be divided into multiple unparallel operations) and series-reduce tasks (each reduce task consists of only one operation). Different from the classical MapReduce scheduling problem, we also assume that all the operations cannot be processed in parallel, and the machine settings are unrelated machines. For solving the extended MapReduce scheduling problem, we establish a mixed-integer programming model with the minimum makespan as the objective function. We then propose a genetic algorithm, a simulated annealing algorithm, and an L-F algorithm to solve this problem. Numerical experiments show that L-F algorithm has better performance in solving this problem.

show abstract