Improving MapReduce Load Balancing in Hadoop

Syue, Fu-Hong; Kshirsagar, Varsha A.; Lo, Shou-Chih

doi:10.1109/fskd.2018.8687158

Cited by 2 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the LAHP algorithm partitions the intermediate key value pairs arbitrarily without considering the statistical distribution of pre-processing and input data. A load balancing mechanism that enhances MapReduce in Hadoop was proposed for mitigating negative impact of data skew on the performance of MapReduce [19]. Data skew has become a typical problem in MapReduce processing for handling data intensive applications.…”

Section: Related Workmentioning

confidence: 99%

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Ajibade¹,

Bakar²,

Aliyu³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Processing huge and complex data to obtain useful information is challenging, even though several big data processing frameworks have been proposed and further enhanced. One of the prominent big data processing frameworks is MapReduce. The main concept of MapReduce framework relies on distributed and parallel processing. However, MapReduce framework is facing serious performance degradations due to the slow execution of certain tasks type called stragglers. Failing to handle stragglers causes delay and affects the overall job execution time. Meanwhile, several straggler reduction techniques have been proposed to improve the MapReduce performance. This study provides a comprehensive and qualitative review of the different existing straggler mitigation solutions. In addition, a taxonomy of the available straggler mitigation solutions is presented. Critical research issues and future research directions are identified and discussed to guide researchers and scholars.

show abstract

Section: Related Workmentioning

confidence: 99%