EPPADS: An Enhanced Phase-Based Performance-Aware Dynamic Scheduler for High Job Execution Performance in Large Scale Clusters

Hamandawana, Prince; Mativenga, Ronnie; Kwon, Se Jin; Chung, Tae-Sun

doi:10.1007/978-3-030-18576-3_9

Cited by 2 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A lot of prior research works have already put a lot of effort in trying to improve the job processing performance in clusters provisioned with heterogeneous nodes [6]- [10], [17]- [20]. This heterogeneity, coupled with fluctuating workload conditions, leads to degraded job processing performance and this can become worse in large scale data processing clusters which handles exabytes of data per day.…”

Section: B Performance-aware Dynamic Schedulingmentioning

confidence: 99%

“…Therefore, effective energy saving approaches needs to be designed to solve the power consumption problems faced in large scale data processing clusters [11]- [16]. Previous works [17], tried to resolve the performance penalty caused by node instrumentation, but it did not resolve the power consumption problem faced in large scale clusters. To address the discussed limitations of the existing approaches, we propose an energy efficient Enhanced Phase-based Performance Aware Dynamic Scheduler (EPPADS).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

et al. 2019

Self Cite

View full text Add to dashboard Cite

Many prior works have investigated on how to increase the job processing performance and energy efficient computing in large scale clusters. However, they employ serialized scheduling approaches encompassed with task straggler ''hunting'' techniques which launches speculative tasks after detecting slow tasks. These slow tasks are detected through node instrumentation which collects system level information whilst tracking the task execution progress. Such approaches are however detrimental towards achieving maximum processing performance and preserving cluster energy as they increase communication overheads. In this paper, we observe that node instrumentation and serialized scheduling in existing works does not only degrade the job processing performance, but also increase cluster energy consumption. To alleviate this, we propose EPPADS, a light-weight scheduler which eradicates the need for instrumentation modules for job scheduling purposes. EPPADS schedules tasks in two stages, the slow-start phase (SSP) and accelerate phase (AccP). The SSP schedules initial tasks in the queue using baseline FIFO scheduling and records the initial execution times of the processing nodes, whilst tagging the effective and straggling nodes. The AccP uses the initial execution times to compute the processing nodes task distribution ratio of remaining tasks and schedules them in parallel using a single scheduling I/O, boosting up the processing performance. To amortize the computing energy costs, EPPADS implements a power management module that coordinates with the scheduling module and leverage on node tagging information, to place nodes in two different power transition pools, i.e., high and low state power pools. A single power transition signal per pool is then broadcasted to lower or raise the energy state in the low-power state pool and high-power state pool. Our evaluation using a Hadoop cluster shows that EPPADS achieves 30% and 22% performance improvement and 15% to 20% energy savings as compared to the FIFO and DynMon schedulers, respectively. INDEX TERMSDistributed processing, energy efficiency, scheduling. PRINCE HAMANDAWANA received the B.Sc. degree (Hons.

show abstract

Section: B Performance-aware Dynamic Schedulingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

2020

View full text Add to dashboard Cite

Cloud computing systems are splitting compute and data intensive jobs into smaller tasks to execute them in a parallel manner using clusters to improve execution time. However, such systems at increasing scale are exposed to stragglers, whereby abnormally slow running tasks executing within a job substantially affect job performance completion. Such stragglers are a direct threat towards attaining fast execution of data intensive jobs within cloud computing. Researchers have proposed an assortment of different mechanisms, frameworks, and management techniques to detect and mitigate stragglers both proactively and reactively. In this paper, we present a comprehensive review of straggler management techniques within large-scale cloud datacenters. We provide a detailed taxonomy of straggler causes, as well as proposed management and mitigation techniques based on straggler characteristics and properties. From this systematic review, we outline several outstanding challenges and potential directions of possible future work for straggler research.

show abstract

EPPADS: An Enhanced Phase-Based Performance-Aware Dynamic Scheduler for High Job Execution Performance in Large Scale Clusters

Cited by 2 publications

References 13 publications

Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

Towards an Energy Efficient Computing With Coordinated Performance-Aware Scheduling in Large Scale Data Clusters

Tails in the cloud: a survey and taxonomy of straggler management within large-scale cloud data centres

Contact Info

Product

Resources

About