MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

Nita, Mihaela-Catalina; Pop, Florin; Voicu, Cristiana; Dobre, Ciprian; Xhafa, Fatos

doi:10.1007/s10586-015-0454-8

Cited by 20 publications

(16 citation statements)

References 36 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The technique depends on the presence of two operations, each of which receives a message that is known as the heartbeat message. Once a job is queued by the scheduler, a heartbeat is received that consequently lead to triggering the proper routing to assign free resource to the queued job, as a result the mean completion time of the job is decreased (Nagina & Dhingra, 2016) 3.10 MOMTH:"Multi-objective scheduling algorithm of many tasks in Hadoop" (Nita, Pop, Voicu, Dobre, & Xhafa, 2015) MOMTH is a Multi-objective/Multi-constrains Hadoop job scheduling approach, aims to improve the processing of big data. MOMTH algorithm takes into consideration two objectives, first one of which is an optimal workload of the cluster and the second objective is avoiding resources contention.…”

Section: Resource Aware Schedulermentioning

confidence: 99%

See 1 more Smart Citation

Hadoop MapReduce Job Scheduling Algorithms Survey and Use Cases

Abdallat¹,

Alahmad²,

amimi³

et al. 2019

MAS

View full text Add to dashboard Cite

Data is the fastest growing asset in the 21st century, extracting insights is becoming of the essence as the traditional ecosystems are incapable to process the resulting amounts, complying with different structural levels, and is rapidly produced. Along this paradigm, the need for processing mostly real time data among other factors highlights the need for optimized Job Scheduling Algorithms, which is the interest of this paper. It is one of the most important aspects to guarantee an efficient processing ecosystem with minimal execution time, while exploiting the available resources taking into consideration granting all the users a fair share of the dedicated resources. Through this work, we lay some needed background on the Hadoop MapReduce framework. We run a comparative analysis on different algorithms that are classified on different criteria. The light is shed on different classifications: Cluster Environment, Job Allocation Strategy, Optimization Strategy, and Metrics of Quality. We, also, construct use cases to showcase the characteristics of selected Job Scheduling Algorithms, then we present a comparative display featuring the details for the use cases.

show abstract

Section: Resource Aware Schedulermentioning

confidence: 99%

“…With respect to two constraints, for example deadline and budget. This Approach provides a performance measurements that are similar to the ones of FIFO and Fair schedulers (Nita et al, 2015). As a general rule the two objectives are users' related objective and resources related objective (Nagina & Dhingra, 2016).…”

Section: Resource Aware Schedulermentioning

confidence: 99%

Hadoop MapReduce Job Scheduling Algorithms Survey and Use Cases

Abdallat¹,

Alahmad²,

amimi³

et al. 2019

MAS

View full text Add to dashboard Cite

show abstract

“…It constitutes the issue of how to identify and cache main data which are separated from unorganized information, proficiently [1,3].…”

Section: Storage Of Massive Quantity Of Informationmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 1 more Smart Citation

Job Scheduling in Big Data-A Survey

Seethalakshmi¹,

Govindasamy²,

Akila³

2018

J Inform Tech Softw Eng

View full text Add to dashboard Cite

IntroductionCurrently, the phrase Big Data has become most fashionable in IT region. It refers to a broad area of dataset which are tough to be maintained by traditional functions [1]. Big Data can be used in Economics and Commerce, Finance, Electronic shopping, Medicare, Astrophysics, Oceanology, Manufacturing and numerous different areas. These datasets are most difficult. The data size is increasing exponentially day by day in extremely huge quantity. As information is rising in capacity, in variety and with huge speed, it also increases the complexities in handling it. Big Data is a developing area. It has a lot of investigation problems and objections to address. The major problems in Big Data are: i) Managing data quantity, ii) Analysis of Big Data, iii) Privacy of information, iv) Holding of massive quantity of information, v) Information visualization, vi) Job scheduling in Big Data, vii) Fault tolerance. Research Methodology Managing information quantityThe huge quantity of information/data imminent from various areas of education such as genetics, astrophysics, weather forecasting, etc makes it extremely hard for the biologists to handle [1,2]. Analysis of big dataIt is hard to diagnose Big Data due to inhomogeneous and incompleteness of information. Composed information can be in various methods, diversity and structure [3]. Privacy of information in the context of big dataThere is a general fear regarding to the improper utilization of individual information, mainly through connecting the information from numerous resources. Handling secrecy is both a scientific and a Sociological issue [3]. Storage of massive quantity of informationIt constitutes the issue of how to identify and cache main data which are separated from unorganized information, proficiently [1,3]. AbstractA huge amount of information (Information in the unit of Exabyte or Zettabyte) is called Big Data. To quantify such a large amount of data and store electronically is not easy. Hadoop system is used to handle these large data sets. To collect Big Data according to the request, Map Reduce program is used. In order to achieve greater performance, Big Data requires proper scheduling. To reduce starvation and increase the use of resource and also to assign the jobs for available resources, the scheduling technique is used. The Performance can be increased by implementing deadline constraints on jobs. The goal of the paper is to study and analyze various scheduling algorithms for better performance.

show abstract

Context‐aware scheduling in MapReduce: a compact review

Idris

Hussain

Ali

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYIt is a fact that the attention of research community in computer science, business executives, and decision makers is drastically drawn by big data. As the volume of data becomes bigger, it needs performance-oriented data-intensive processing frameworks such as MapReduce, which can scale computation on large commodity clusters. Hadoop MapReduce processes data in Hadoop Distributed File System as jobs scheduled according to YARN fair scheduler and capacity scheduler. However, with advancement and dynamic changes in hardware and operating environments, the performance of clusters is greatly affected. Various efforts in literature have been made to address the issues of heterogeneity (i.e., clusters consisting of virtual machines and machines with different hardware), network communication, data locality, better resource utilization, and run-time scheduling. In this paper, we present a survey to discuss various research efforts made so far to improve Hadoop MapReduce scheduling. We classify scheduling algorithms and techniques proposed in the literature so far based on their addressing areas and present a taxonomy. Furthermore, we also discuss various aspects of open issues and challenges in the scheduling of MapReduce to improve its performance.

show abstract

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

Cited by 20 publications

References 36 publications

Hadoop MapReduce Job Scheduling Algorithms Survey and Use Cases

Hadoop MapReduce Job Scheduling Algorithms Survey and Use Cases

Job Scheduling in Big Data-A Survey

Context‐aware scheduling in MapReduce: a compact review

Contact Info

Product

Resources

About