Handling the uncertainty in resource performance for executing workflow applications in clouds

Abstract

Execution of workflow applications in Cloud environments involves many uncertainties because of elastic resource provisioning and unstable performance of multitenant virtual machines (VM) instances over time. These uncertainties are usually either neglected by existing researches, or modeled with some probability distribution function. To address this gap, we extend a multi-objective workflow scheduling algorithm (MOHEFT) in two directions: (1) to deal with the dynamic nature of Cloud environments offering a potentially infinite amount of on-demand resources, and (2) to consider robustness as an objective that mitigates the variability in VM performance over time. Our new robust model, called R-MOHEFT, considers uncertainty in processing times of workflow activities without a precise estimation or known distribution function within an uncertainty interval. We approach this scheduling problem as a three-objective optimisation that considers makespan, monetary cost, and robustness as simultaneous objectives of a commercial Cloud environment. Our new algorithm is able to estimate the Pareto optimal set of scheduling solutions that resist against fluctuations in processing times three times better than its MOHEFT predecessor, with a tradeoff of only 15% worse Pareto frontier. R-MOHEFT's hypervolume suffers by only 5% to 16%, compared to the MOHEFT's drawback of 38% to surprisingly 87%, when the processing time fluctuates up to its double value. CCS Concepts •Theory of computation → Self-organization; Distributed computing models; •Computing methodologies → Self-organization; •Computer systems organization → Cloud computing;

A Robust Algorithm for Deadline Constrained Scheduling in IaaS Cloud Environment

“…• Scheduling workflows on IaaS Clouds involves many uncertainties due to the variable and unpredictable performance of Cloud resources. These uncertainties are modeled by probability distribution functions in past researches or totally ignored in some cases [6].…”

“…It has been argued that many real life situations have insufficient information to enable the characterization of the probability distribution functions for all random parameters required for scheduling [6]. This has brought forth other approaches.…”

“…The DET strategy however ignored communication time between tasks and both stategies did not consider the uncertainty of task runtime estimations for scheduling decisions. The uncertainty handling strategy in this work was inspired by the work by Fard et al which considered the knowledge of the uncertainty interval for the proposed Robust MO-HEFT algorithm [6]. They approached the scheduling problem as a multi-objective optimization problem which differs from our approach, a cost optimized, deadline-constrained heuristic-based scheduling problem.…”

“…The minimum and maximum values possible are modeled as the lower and upper bounds of the task execution time respectively. Similar to the work in [6], we also assume that the probability function of the distribution of the interval of uncertainty is unknown. Subsequently, our cost optimized deadline-constrained algorithm uses this model to generate a robust schedule that guarantees to meet the user specified deadlines at the least possible cost depending on the interval of uncertainty.…”

A Simplified Model for Simulating the Execution of a Workflow in Cloud

“…Most common simulators allow users to create a virtual data center considering their latest computing, networking, energy, or cost requirements. However, although they can simulate an elastic Cloud data center, simulators usually neglect the Cloud performance fluctuation and uncertainty [10], which can lead to wrong estimation. This aspect is especially important for workflow executions, as they consist of a high number of data and control flow dependencies [15] that further affect their overall performance without any correlation [18].…”