A stochastic model for robust resource allocation in heterogeneous parallel and distributed computing systems

Smith, Jay; Siegel, Howard Jay; Maciejewski, Anthony A.

doi:10.1109/ipdps.2008.4536431

Cited by 7 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, we see a growing interest in [ Many classical optimization methods, such as opportunistic load balance, minimum execution time, and minimum completion time, are described in [11]. Several heuristics, such as sufferage, min-min, max-min, and auction have been proposed in the past [31].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Workflow Scheduling on Cloud Computing Platforms with IterativeOrdinal Optimization

Zhang

Cao

Hwang

et al. 2015

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

The scheduling of multitask jobs on clouds is an NP-hard problem. The problem becomes even worse when complex workflows are executed on elastic clouds, such as Amazon EC2 or IBM RC2. The main difficulty lies in the large search space and high overhead of generating optimal schedules, especially for real-time applications with dynamic workloads. In this work, a new iterative ordinal optimization (IOO) method is proposed. The ordinal optimization method is applied in each iteration to achieve suboptimal schedules. IOO aims at generating more efficient schedules from a global perspective over a long period. We prove through overhead analysis the advantages in time and space efficiency in using the IOO method. The IOO method is designed to adapt to system dynamism to yield suboptimal performance.In cloud experiments on IBM RC2 cloud, we execute 20,000 tasks in LIGO (Laser Interferometer Gravitational-wave Observatory) verification workflow on 128 virtual machines. The IOO schedule is generated in less than 1,000 seconds, while using the Monte Carlo simulation takes 27.6 hours, 100 times longer to yield an optimal schedule. The IOO-optimized schedule results in a throughput of 1,100 tasks/sec with 7 GB memory demand, compared with 60% decrease in throughput and 70% increase in memory demand in using the Monte Carlo method. Our LIGO experimental results clearly demonstrate the advantage of using the IOO-based workflow scheduling over the traditional blind-pick, ordinal optimization, or Monte Carlo methods. These numerical results are also validated by the theoretical complexity and overhead analysis provided.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Dogan and Özgüner [11] developed a matching and scheduling algorithm. Smith, Siegel, and Maciejewski [31] have proposed robust static resource allocation for distributed computing systems operating under imposed QoS constraints. None of these methods investigated the profiles and runtime system performance.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Workflow Scheduling on Cloud Computing Platforms with IterativeOrdinal Optimization

Zhang

Cao

Hwang

et al. 2015

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

show abstract

“…Moretti [24] suggested the All-Pairs to improve usability, performance, and efficiency of a campus grid. Smith et al [30] proposed a robust static resource allocation for distributed computing systems operating under imposed QoS constraints.…”

Section: Related Workmentioning

confidence: 99%

Evolutionary Scheduling of Dynamic Multitasking Workloads for Big-Data Analytics in Elastic Cloud

Zhang

Cao

Tan

et al. 2014

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

Scheduling of dynamic and multitasking workloads for big-data analytics is a challenging issue, as it requires a significant amount of parameter sweeping and iterations. Therefore, real-time scheduling becomes essential to increase the throughput of many-task computing. The difficulty lies in obtaining a series of optimal yet responsive schedules. In dynamic scenarios, such as virtual clusters in cloud, scheduling must be processed fast enough to keep pace with the unpredictable fluctuations in the workloads to optimize the overall system performance. In this paper, ordinal optimization using rough models and fast simulation is introduced to obtain suboptimal solutions in a much shorter timeframe. While the scheduling solution for each period may not be the best, ordinal optimization can be processed fast in an iterative and evolutionary way to capture the details of big-data workload dynamism. Experimental results show that our evolutionary approach compared with existing methods, such as Monte Carlo and Blind Pick, can achieve higher overall average scheduling performance, such as throughput, in real-world applications with dynamic workloads. Furthermore, performance improvement is seen by implementing an optimal computing budget allocating method that smartly allocates computing cycles to the most promising schedules.

show abstract

Proposing an Architecture for Scientific Workflow Management System in Cloud

Samandi¹,

Mukhopadhyay²

2017

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

A stochastic model for robust resource allocation in heterogeneous parallel and distributed computing systems

Abstract: Abstract

Cited by 7 publications

References 11 publications

Adaptive Workflow Scheduling on Cloud Computing Platforms with IterativeOrdinal Optimization

Adaptive Workflow Scheduling on Cloud Computing Platforms with IterativeOrdinal Optimization

Evolutionary Scheduling of Dynamic Multitasking Workloads for Big-Data Analytics in Elastic Cloud

Proposing an Architecture for Scientific Workflow Management System in Cloud

Contact Info

Product

Resources

About