AEGEUS++: an energy-aware online partition skew mitigation algorithm for mapreduce in cloud

Kumaresan, Vimalkumar; Baskaran, R.; Dhavachelvan, P.

doi:10.1007/s10586-017-1044-8

Cited by 4 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the proposed approach, Hadoop jobs are not considered for preemption. AEGEUS++ proposed an opportunistic frequency tuning algorithm for energy optimization in Hadoop cluster which focus on homogenous environment [20]. AEGEUS++ focus on optimizing the make-span and energy utilization problem.…”

Section: Related Workmentioning

confidence: 99%

SAGS: A SLA-Aware Green Scheduling in Heterogeneous Cloud Using Hadoop YARN

Balagoni¹,

Rao²

2018

IJIES

View full text Add to dashboard Cite

The Apache Hadoop with cloud had become an emerging and popular service. Irrespective of its huge dominance in large scale data processing, it has challenges yet to be addressed. The primary challenges in yarn scheduler are the abilities to automate and control the resource allocation to different workloads in order to meet the deadline-based Service Level Agreement (SLA) in the cloud environment with optimal energy consumption. Our study with the Hadoop YARN addresses this problem in a controlled homogeneous environment. In cloud datacenters, heterogeneity had become a normal phenomenon. Hence, this paper proposes the problem of energy-aware heterogeneous Hadoop Yarn cloud with deadline based SLA. We proposed a SLA-Aware Green Scheduling (SAGS), a Dynamic Voltage/Frequency Scaling (DVFS) based approach along with SLA-Aware scheduling algorithm in the heterogeneous environment. We evaluated SAGS by using benchmark datasets and, compared its performance with previously proposed solutions. Our observation with experimental results shows that, the proposed approach outperforms existing approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

SAGS: A SLA-Aware Green Scheduling in Heterogeneous Cloud Using Hadoop YARN

Balagoni¹,

Rao²

2018

IJIES

View full text Add to dashboard Cite

show abstract

“…Although this strategy improves performance of large data, it imposes high overhead, mainly on small amounts of data [9]. Recently, some approaches [2,[10][11][12] have been proposed by using prediction techniques to find the partition size even before the completion of the map tasks. These solutions dynamically allocate resources for reduce tasks according to their estimated value.…”

Section: Introductionmentioning

confidence: 99%

MAPSkew: Metaheuristic Approaches for Partitioning Skew in MapReduce

et al. 2018

View full text Add to dashboard Cite

MapReduce is a parallel computing model in which a large dataset is split into smaller parts and executed on multiple machines. Due to its simplicity, MapReduce has been widely used in various applications domains. MapReduce can significantly reduce the processing time of a large amount of data by dividing the dataset into smaller parts and processing them in parallel in multiple machines. However, when data are not uniformly distributed, we have the so called partitioning skew, where the allocation of tasks to machines becomes unbalanced, either by the distribution function splitting the dataset unevenly or because a part of the data is more complex and requires greater computational effort. To solve this problem, we propose an approach based on metaheuristics. For evaluating purposes, three metaheuristics were implemented: Simulated Annealing, Local Beam Search and Stochastic Beam Search. Our experimental evaluation, using a MapReduce implementation of the Bron-Kerbosch Clique Algorithm, shows that the proposed method can find good partitionings while better balancing data among machines.

show abstract