Co-scheduling compute-intensive tasks on a network of workstations: model and algorithms

Atallah, Mikhail J.; Lock, C.; Marinescu, Dan C.; Siegel, Howard Jay; Casavant, T.L.

doi:10.1109/icdcs.1991.148687

Cited by 11 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many schedule algorithms has been proposed [11,12,13,16,17]. [16,17] focus on scheduling tasks on heterogeneous hardware, and [11,12,13] focus on the system performance under diverse workload. The heterogeneity of workloads is also in our assumptions.…”

Section: Related Workmentioning

confidence: 99%

A Dynamic MapReduce Scheduler for Heterogeneous Workloads

Tian

Zhou

et al. 2009

2009 Eighth International Conference on Grid and Cooperative Computing

125

View full text Add to dashboard Cite

MapReduce is an important programming model for building data centers containing ten of thousands of nodes. In a practical data center of that scale, it is a common case that I/Obound jobs and CPU-bound jobs, which demand different resources, run simultaneously in the same cluster. In the MapReduce framework, parallelization of these two kinds of job has not been concerned. In this paper, we give a new view of the MapReduce model, and classify the MapReduce workloads into three categories based on their CPU and I/O utilization. With workload classification, we design a new dynamic MapReduce workload predict mechanism, MR-Predict, which detects the workload type on the fly. We propose a Triple-Queue Scheduler based on the MR-Predict mechanism. The Triple-Queue scheduler could improve the usage of both CPU and disk I/O resources under heterogeneous workloads. And it could improve the Hadoop throughput by about 30% under heterogeneous workloads.

show abstract

Section: Related Workmentioning

confidence: 99%

A Dynamic MapReduce Scheduler for Heterogeneous Workloads

Tian

Zhou

et al. 2009

2009 Eighth International Conference on Grid and Cooperative Computing

125

View full text Add to dashboard Cite

show abstract

“…Co-scheduling is an alternative to gang scheduling [8,9]. In the case of co-scheduling, the dynamics of synchronization requirements of the application is taken into account; only those members of the process group which need to communicate with one another are scheduled concurrently.…”

Section: Overviewmentioning

confidence: 99%

Dynamic scheduling of process groups

Wang

Marinescu

Carbunar

1998

Concurrency: Pract. Exper.

View full text Add to dashboard Cite

In this paper we introduce the concept of temporal locality of commUnici:Ltion for process groups. Empirical evidence suggests that, once a member of a process group starts to communicate with other processes in the group, it will continue to do so, while an independent process will maintain its state of isolation for some time. Other instances of inertial behavior of programs are known. Temporal and spatIal locality of reference are example of inertial behavior of programs, exploited by hierarchical storage systems; once a block of information (program or data) is brought into faster storage, It 1s very likely that it will be referenced again within a short time frame.The temporal locality of communlcation can be used to schedule concurrently multiple process groups. When process groups exhibit temporal locality of commun..ic;:Liion, this information can be used to hide the latency of paging and I/O operations, to perform dynamic scheduling to reduce processor fragmentation, and to identify optimal instances of time for checkpointing of process groups.

show abstract

“…Atallah et al [2] redefine coscheduling from an effective speedup perspective. In their study, they first give a concept called "duty cycle," which is defined as the ratio of cycles a workstation commits to local jobs to the number of cycles available for parallel processes.…”

Section: Related Workmentioning

confidence: 99%

Coordinating Parallel Processes on Networks of Workstations

Zhang

1997

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

Co-scheduling compute-intensive tasks on a network of workstations: model and algorithms

Cited by 11 publications

References 8 publications

A Dynamic MapReduce Scheduler for Heterogeneous Workloads

A Dynamic MapReduce Scheduler for Heterogeneous Workloads

Dynamic scheduling of process groups

Coordinating Parallel Processes on Networks of Workstations

Contact Info

Product

Resources

About