Monotasks

Ousterhout, Kay; Canel, Christopher; Ratnasamy, Sylvia; Shenker, Scott

doi:10.1145/3132747.3132766

Cited by 62 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Without controlling the disk I/O, some tasks will exhaust disk I/O, which would incur disk blocking of other tasks and increase their completion time. Improved job completion time after implementing a disk optimization approach represents a best‐case scenario 67 . DRF uses the concept of a “dominant resource” to compare multidimensional resources.…”

Section: Our Enhanced Fairer Resource Allocation Strategymentioning

confidence: 99%

An efficient deadline constrained and data locality aware dynamic scheduling framework for multitenancy clouds

Yang

Grundy

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Scheduling and resource allocation in clouds is used to harness the power of the underlying resource pool. Service providers can meet quality of service (QoS) requirements of tenants specified in Service Level Agreements. Improving resource allocation ensures that all tenants will receive fairer access to system resources, which improves overall utilization and throughput. Real‐time applications and services require critical deadlines in order to guarantee QoS. A growing number of data‐intensive applications drive the optimization of scheduling through utilizing data locality in which the scheduler locates a task and ensures the task's relevant data to be on the same server. Choosing suitable scheduling mechanisms for running applications that support multitenancy has consistently been a major challenge. This work proposes a new adaptive Deadline constrained and Data locality aware Dynamic Scheduling Framework “ 3DSF“ that orchestrates different schedulers based on varied requirements. This framework considers tenants' deadline‐based QoS requirements, cloud system's performance and a method of resource allocation to improve resource utilization, system throughput and reduce jobs' completion time. 3DSF contains: (a) a real‐time, preemptive, deadline constrained job scheduler, (b) an optimized data locality aware scheduler, (c) an improved Dominant Resource Fairness greedy resource allocation approach, and (d) an adaptive suite to integrate above‐mentioned schedulers together.

show abstract

Section: Our Enhanced Fairer Resource Allocation Strategymentioning

confidence: 99%

An efficient deadline constrained and data locality aware dynamic scheduling framework for multitenancy clouds

Yang

Grundy

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…In shared-memory systems task granularity is often discussed in terms of a target task duration [4], [6]. Some researchers have proposed even more extreme task granularity, k ≫ l, coining the term "tiny tasks" [39].…”

Section: Introduction To Tiny Tasksmentioning

confidence: 99%

The Tiny-Tasks Granularity Trade-Off: Balancing Overhead Versus Performance in Parallel Systems

Bora

Walker

Fidler

2023

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Models of parallel processing systems typically assume that one has l workers and jobs are split into an equal number of k = l tasks. Splitting jobs into k > l smaller tasks, i.e. using "tiny tasks", can yield performance and stability improvements because it reduces the variance in the amount of work assigned to each worker, but as k increases, the overhead involved in scheduling and managing the tasks begins to overtake the performance benefit. We perform extensive experiments on the effects of task granularity on an Apache Spark cluster, and based on these, developed a four-parameter model for task and job overhead that, in simulation, produces sojourn time distributions that match those of the real system. We also present analytical results which illustrate how using tiny tasks improves the stability region of split-merge systems, and analytical bounds on the sojourn and waiting time distributions of both split-merge and single-queue fork-join systems with tiny tasks. Finally we combine the overhead model with the analytical models to produce an analytical approximation to the sojourn and waiting time distributions of systems with tiny tasks which include overhead. We also perform analogous tiny-tasks experiments on a hybrid multi-processor shared memory system based on MPI and OpenMP which has no loadbalancing between nodes. Though no longer strict analytical bounds, our analytical approximations with overhead matched both the Spark and MPI/OpenMP experimental results very well.

show abstract

“…These systems have become popular tools for workloads that range from data aggregation and search to relational queries, graph processing, and machine learning 7‐9 . Jobs from these diverse domains stress different resources, while the resource demands typically also fluctuate significantly over the runtime of jobs 2,10‐12 . Therefore, multiple jobs usually share cluster resources without isolation, so they can benefit from statistical multiplexing 2,13,14 .…”

Section: Introductionmentioning

confidence: 99%

Mary, Hugo, and Hugo*: Learning to schedule distributed data‐parallel processing jobs on shared clusters

Thamsen

Beilharz

Tran

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Distributed data‐parallel processing systems like MapReduce, Spark, and Flink are popular for analyzing large datasets using cluster resources. Resource management systems like YARN or Mesos in turn allow multiple data‐parallel processing jobs to share cluster resources in temporary containers. Often, the containers do not isolate resource usage to achieve high degrees of overall resource utilization despite overprovisioning and the often fluctuating utilization of specific jobs. However, some combinations of jobs utilize resources better and interfere less with each other when running on the same shared nodes than others. This article presents an approach for improving the resource utilization and job throughput when scheduling recurring distributed data‐parallel processing jobs in shared clusters. The approach is based on reinforcement learning and a measure of co‐location goodness to have cluster schedulers learn over time which jobs are best executed together on shared resources. We evaluated this approach over the last years with three prototype schedulers that build on each other: Mary, Hugo, and Hugo*. For the evaluation we used exemplary Flink and Spark jobs from different application domains and clusters of commodity nodes managed by YARN. The results of these experiments show that our approach can increase resource utilization and job throughput significantly.

show abstract

Monotasks

Cited by 62 publications

References 13 publications

An efficient deadline constrained and data locality aware dynamic scheduling framework for multitenancy clouds

An efficient deadline constrained and data locality aware dynamic scheduling framework for multitenancy clouds

The Tiny-Tasks Granularity Trade-Off: Balancing Overhead Versus Performance in Parallel Systems

Mary, Hugo, and Hugo*: Learning to schedule distributed data‐parallel processing jobs on shared clusters

Contact Info

Product

Resources

About