Mitigating stragglers to avoid QoS violation for time-critical applications through dynamic server blacklisting

Ouyang, Xue; Wang, Changjian; Xu, Jie

doi:10.1016/j.future.2019.07.017

Cited by 8 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A Dynamic Server Blacklisting (DSB) framework was proposed for lessening stragglers to evade Quality of Service violation for time-sensitive applications [24]. The straggler task may occur due to one or more of the following reasons: heterogeneous hardware configuration, resource contention and www.ijacsa.thesai.org so on.…”

Section: Related Workmentioning

confidence: 99%

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Ajibade¹,

Bakar²,

Aliyu³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Processing huge and complex data to obtain useful information is challenging, even though several big data processing frameworks have been proposed and further enhanced. One of the prominent big data processing frameworks is MapReduce. The main concept of MapReduce framework relies on distributed and parallel processing. However, MapReduce framework is facing serious performance degradations due to the slow execution of certain tasks type called stragglers. Failing to handle stragglers causes delay and affects the overall job execution time. Meanwhile, several straggler reduction techniques have been proposed to improve the MapReduce performance. This study provides a comprehensive and qualitative review of the different existing straggler mitigation solutions. In addition, a taxonomy of the available straggler mitigation solutions is presented. Critical research issues and future research directions are identified and discussed to guide researchers and scholars.

show abstract

Section: Related Workmentioning

confidence: 99%

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Ajibade¹,

Bakar²,

Aliyu³

et al. 2022

IJACSA

View full text Add to dashboard Cite

show abstract

“…Many researchers have attempted to avoid stragglers through machine learning approaches. e poor performing nodes are identified and blacklisted [30,31] during the task scheduling phase. ese techniques again lead to resource wastage, as they are not able to participate in the execution as stragglers are mainly nonpersistent.…”

Section: Security and Communication Networkmentioning

confidence: 99%

Collaborative Learning Based Straggler Prevention in Large-Scale Distributed Computing Framework

Deshmukh

Rao

Shabaz

2021

Security and Communication Networks

View full text Add to dashboard Cite

Modern big data applications tend to prefer a cluster computing approach as they are linked to the distributed computing framework that serves users jobs as per demand. It performs rapid processing of tasks by subdividing them into tasks that execute in parallel. Because of the complex environment, hardware and software issues, tasks might run slowly leading to delayed job completion, and such phenomena are also known as stragglers. The performance improvement of distributed computing framework is a bottleneck by straggling nodes due to various factors like shared resources, heavy system load, or hardware issues leading to the prolonged job execution time. Many state-of-the-art approaches use independent models per node and workload. With increased nodes and workloads, the number of models would increase, and even with large numbers of nodes. Not every node would be able to capture the stragglers as there might not be sufficient training data available of straggler patterns, yielding suboptimal straggler prediction. To alleviate such problems, we propose a novel collaborative learning-based approach for straggler prediction, the alternate direction method of multipliers (ADMM), which is resource-efficient and learns how to efficiently deal with mitigating stragglers without moving data to a centralized location. The proposed framework shares information among the various models, allowing us to use larger training data and bring training time down by avoiding data transfer. We rigorously evaluate the proposed method on various datasets with high accuracy results.

show abstract

“…However, even with such measures in place, within large-scale cloud datacenters a sub-set of tasks within a job will manifest as stragglers [86] [87]. In this context, a straggler is defined as task which execute abnormally slow in comparison to the average task duration within a job [12].…”

Section: Straggler Definition and Impactmentioning

confidence: 99%

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

2020

Self Cite

View full text Add to dashboard Cite

Cloud computing systems are splitting compute and data intensive jobs into smaller tasks to execute them in a parallel manner using clusters to improve execution time. However, such systems at increasing scale are exposed to stragglers, whereby abnormally slow running tasks executing within a job substantially affect job performance completion. Such stragglers are a direct threat towards attaining fast execution of data intensive jobs within cloud computing. Researchers have proposed an assortment of different mechanisms, frameworks, and management techniques to detect and mitigate stragglers both proactively and reactively. In this paper, we present a comprehensive review of straggler management techniques within large-scale cloud datacenters. We provide a detailed taxonomy of straggler causes, as well as proposed management and mitigation techniques based on straggler characteristics and properties. From this systematic review, we outline several outstanding challenges and potential directions of possible future work for straggler research.

show abstract

Mitigating stragglers to avoid QoS violation for time-critical applications through dynamic server blacklisting

Abstract: This is a repository copy of Mitigating stragglers to avoid QoS violation for time-critical applications through dynamic server blacklisting.

Cited by 8 publications

References 28 publications

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Collaborative Learning Based Straggler Prevention in Large-Scale Distributed Computing Framework

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

Contact Info

Product

Resources

About