Performance enhancement of Hadoop MapReduce framework for analyzing BigData

Prabhu, S.; Rodrigues, Anisha P; S, Guru Prasad M; Nagesh, H. R.

doi:10.1109/icecct.2015.7226049

Cited by 9 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The related work for optimizing MapReduce performance is mainly data locality and tuning of Hadoop [9] parameters in reliable and homogeneous cluster environments. Prabhu et al [10] reported a simple way to find optimal values for the three categories of 180 parameters: the core-related, MapReduce-related, and HDFS-related. The tuning was fully based on trial and error.…”

Section: Related Workmentioning

confidence: 99%

The Performance Optimization of Big Data Processing by Adaptive MapReduce Workflow

Tang

2022

IEEE Access

View full text Add to dashboard Cite

The discussion context of this paper is big data processing of MapReduce by volunteer computing in dynamic and opportunistic environments. This paper conducts a series of simulations to explore the relationship between the overall performance of volunteer overlays responding to different workload of big data problems. The discovery from the simulations includes some optimization points in overlay size, going over which by adding more volunteers brings little benefit for the overall performance. Based on the discovery of optimization points, this paper proposes a bootstrapping protocol, which can adapt volunteers into variable-sizes overlays, enabling workflow of single-round MapReduce or multipleround MapReduce, and a single or multiple overlays for each round. The variable overlays aim to create adaptive workflow during MapReduce processing, so that the optimization points can be caught. As another benefit, the unnecessary computing-capacities can be released during computing when the optimization points are reached. The case study shows a few optimization workflows that are formed by the proposed bootstrapping protocol to process the big data cases. The workflows lead to the optimization points and dynamically balance the workload at the same time. The experiment results have demonstrated that the optimization strategies have either achieved 36% or 71% higher performance than the plain MapReduce workflow and minimized the use of computing resources by releasing 12.5% to 75% volunteers during computing, where the original plain MapReduce must hold all the volunteers to the end of computing. The extensibility of the simulation parameterization to more diverse real-world applications have been clarified.

show abstract

Section: Related Workmentioning

confidence: 99%

The Performance Optimization of Big Data Processing by Adaptive MapReduce Workflow

Tang

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A few studies [5,14,42,48] have examined the basic models for the extraction of structured data from unstructured information sources and articulated the related challenges with technologies like Hadoop and MapReduce. Another study [49] has also discussed efficient platforms for BDA and identified related performance issues. One more study [50] also proposed a taxonomy and analysis of indexing techniques to organize access to big data under varying conditions.…”

Section: Structured Vs Unstructured Data (Sud 10)mentioning

confidence: 99%

Big Data Redux: New Issues and Challenges Moving Forward

Espinosa

Kaisler

Armour

et al. 2019

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

As of the time of this writing, our HICSS-46 proceedings article has enjoyed over 520 Google Scholar citations. We have published several HICSS proceedings, articles and a book on this subject, but none of them have generated this level of interest. In an effort to update our findings six years later, and to understand what is driving this interest, we have downloaded the first 500 citations to our article and the corresponding citing article, when available. We conducted an in-depth literature review of the articles published in top journals and leading conference proceedings, along with articles with a high volume of citations. This paper provides a brief summary of the key concepts in our original paper and reports on the key aspects of interest we found in our review, and also updates our original paper with new directions for future practice and research in big data and analytics.

show abstract