Client-side straggler-aware I/O scheduler for object-based parallel file systems

Tavakoli, Neda; Dong, Dewen; Chen, Yong

doi:10.1016/j.parco.2018.07.001

Cited by 11 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6) Imbalance: Drishti detects imbalance in time (stragglers) and in transfer size. While the first might be caused by external factors such as network or file system contention, the latter is often related to how an application handles its data [32]- [34]. A common root cause of performance degradation is by having a single process, often rank 0, be responsible for writing and reading data.…”

Section: Infomentioning

confidence: 99%

Drishti: Guiding End-Users in the I/O Optimization Journey

Bez

Ather

Byna

2022

2022 IEEE/ACM International Parallel Data Systems Workshop (PDSW)

View full text Add to dashboard Cite

The complex software and hardware I/O stack of HPC platforms makes it challenging for end-users to obtain superior I/O performance and to understand the root causes of I/O bottlenecks they encounter. Despite the continuous efforts from the community to profile I/O performance and propose new optimization techniques and tuning options for improving the performance, there is still a translation gap between profiling and tuning. In this paper, we propose Drishti, a solution to guide scientists in optimizing I/O in their applications by detecting typical I/O performance pitfalls and providing recommendations. We illustrate its applicability in two case studies and evaluate its robustness and performance by summarizing the issues detected in over a hundred thousand Darshan logs collected on the Cori supercomputer at the National Energy Research Scientific Computing Center (NERSC). Drishti can empower end-users and guide them in the I/O optimization journey by shedding some light on everyday I/O performance pitfalls and how to fix them.

show abstract

Section: Infomentioning

confidence: 99%

Drishti: Guiding End-Users in the I/O Optimization Journey

Bez

Ather

Byna

2022

2022 IEEE/ACM International Parallel Data Systems Workshop (PDSW)

View full text Add to dashboard Cite

show abstract

“…The literature reports [45] [61] [72] [102] that the straggler management techniques create several copies of the same task to mitigate the effects of stragglers. Copying a task reserves additional resources such as the disk, memory of CPU time, increasing use of particular resource.…”

Section: Energy Managementmentioning

confidence: 99%

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

2020

View full text Add to dashboard Cite

Cloud computing systems are splitting compute and data intensive jobs into smaller tasks to execute them in a parallel manner using clusters to improve execution time. However, such systems at increasing scale are exposed to stragglers, whereby abnormally slow running tasks executing within a job substantially affect job performance completion. Such stragglers are a direct threat towards attaining fast execution of data intensive jobs within cloud computing. Researchers have proposed an assortment of different mechanisms, frameworks, and management techniques to detect and mitigate stragglers both proactively and reactively. In this paper, we present a comprehensive review of straggler management techniques within large-scale cloud datacenters. We provide a detailed taxonomy of straggler causes, as well as proposed management and mitigation techniques based on straggler characteristics and properties. From this systematic review, we outline several outstanding challenges and potential directions of possible future work for straggler research.

show abstract

“…This section provides a brief overview of the technical aspects of reinforcement learning and its deep version. This technique has several interesting applications in different domains [11], [15].…”

Section: Deep Reinforcement Learning: Backgroundmentioning

confidence: 99%

Deep Reinforcement Learning for Detecting Malicious Websites

Chatterjee,

Namin

2019

Preprint

View full text Add to dashboard Cite

Phishing is the simplest form of cybercrime with the objective of baiting people into giving away delicate information such as individually recognizable data, banking and credit card details, or even credentials and passwords. This type of simple yet most effective cyber-attack is usually launched through emails, phone calls, or instant messages. The credential or private data stolen are then used to get access to critical records of the victims and can result in extensive fraud and monetary loss. Hence, sending malicious messages to victims is a stepping stone of the phishing procedure. A phisher usually setups a deceptive website, where the victims are conned into entering credentials and sensitive information. It is therefore important to detect these types of malicious websites before causing any harmful damages to victims. Inspired by the evolving nature of the phishing websites, this paper introduces a novel approach based on deep reinforcement learning to model and detect malicious URLs. The proposed model is capable of adapting to the dynamic behavior of the phishing websites and thus learn the features associated with phishing website detection.

show abstract

Client-side straggler-aware I/O scheduler for object-based parallel file systems

Cited by 11 publications

References 39 publications

Drishti: Guiding End-Users in the I/O Optimization Journey

Drishti: Guiding End-Users in the I/O Optimization Journey

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

Deep Reinforcement Learning for Detecting Malicious Websites

Contact Info

Product

Resources

About