On-Line, Non-clairvoyant Optimization of Workflow Activity Granularity on Grids

Silva, Rafael Ferreira da; Glatard, Tristan; Desprez, Frédéric

doi:10.1007/978-3-642-40047-6_28

Cited by 7 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Task granularity has been addressed in several research studies for reducing the impact of overheads that may arise when executing scientific workflows in distributed environments, such as the cloud [4,12,21]. Task grouping methods reduce computational granularity by reducing the number of computational activities by grouping fine-grained tasks into course-grained tasks.…”

Section: Related Workmentioning

confidence: 99%

“…The elapsed time between the deadline and the estimated makespan, denoted as the available spare time is calculated according to Equation (11). is distributed proportionally over all levels of the workflow on the basis of runtime of tasks according to Equation (12). Then, is distributed among all tasks of each level proportional to the number of tasks in the corresponding level according to Equation (13).…”

Section: Deadline Distributionmentioning

confidence: 99%

“…For example, dynamic provisioning of resources is not considered in [5][6][7][8], scalability in terms of large number of tasks is not considered in [9], heterogeneity of resources is not considered in [8,10], resource auto-scaling is not considered in [11], data dependencies are not considered in [12] and task clustering technique in [5] is not fully autonomous. Moreover, unlike multiple independent BoTs or single task-based workflows, the concept of using multiple connected and constrained BoTs for reducing the data transfer time is not considered in most existing scheduling algorithms [13,14].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Elastic Scheduling of Scientific Workflows under Deadline Constraints in Cloud Computing Environments

Anwar

Deng

2018

Future Internet

View full text Add to dashboard Cite

Scientific workflow applications are collections of several structured activities and finegrained computational tasks. Scientific workflow scheduling in cloud computing is a challenging research topic due to its distinctive features. In cloud environments, it has become critical to perform efficient task scheduling resulting in reduced scheduling overhead, minimized cost and maximized resource utilization while still meeting the user-specified overall deadline. This paper proposes a strategy, Dynamic Scheduling of Bag of Tasks based workflows (DSB), for scheduling scientific workflows with the aim to minimize financial cost of leasing Virtual Machines (VMs) under a userdefined deadline constraint. The proposed model groups the workflow into Bag of Tasks (BoTs) based on data dependency and priority constraints and thereafter optimizes the allocation and scheduling of BoTs on elastic, heterogeneous and dynamically provisioned cloud resources called VMs in order to attain the proposed method's objectives. The proposed approach considers pay-asyou-go Infrastructure as a Service (IaaS) clouds having inherent features such as elasticity, abundance, heterogeneity and VM provisioning delays. A trace-based simulation using benchmark scientific workflows representing real world applications, demonstrates a significant reduction in workflow computation cost while the workflow deadline is met. The results validate that the proposed model produces better success rates to meet deadlines and cost efficiencies in comparison to adapted state-of-the-art algorithms for similar problems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Deadline Distributionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Elastic Scheduling of Scientific Workflows under Deadline Constraints in Cloud Computing Environments

Anwar

Deng

2018

Future Internet

View full text Add to dashboard Cite

show abstract

“…However such estimates are hard to obtain in production conditions (Ferreira da Silva, Juve, et al, 2013). Therefore, we propose a granularity control algorithm (Ferreira da Silva, Glatard, & Desprez, 2014, 2013a for platforms where such clairvoyant and offline conditions are not realistic. Our method groups tasks when the fineness degree of the application, which takes into account the ratio of shared data and the queuing/round-trip time ratio, becomes higher than a threshold determined from execution traces.…”

Section: Optimizing Task Granularitymentioning

confidence: 99%

Self-Management of Operational Issues for Grid Computing

Silva

Glatard

Desprez

2015

Advances in Systems Analysis, Software Engineering, and High Performance Computing

Self Cite

View full text Add to dashboard Cite

Science gateways, such as the Virtual Imaging Platform (VIP), enable transparent access to distributed computing and storage resources for scientific computations. However, their large scale and the number of middleware systems involved in these gateways lead to many errors and faults. This chapter addresses the autonomic management of workflow executions on science gateways in an online and non-clairvoyant environment, where the platform workload, task costs, and resource characteristics are unknown and not stationary. The chapter describes a general self-management process based on the MAPE-K loop (Monitoring, Analysis, Planning, Execution, and Knowledge) to cope with operational incidents of workflow executions. Then, this process is applied to handle late task executions, task granularities, and unfairness among workflow executions. Experimental results show how the approach achieves a fair quality of service by using control loops that constantly perform online monitoring, analysis, and execution of a set of curative actions.

show abstract

Controlling fairness and task granularity in distributed, online, non‐clairvoyant workflow executions

Silva

Glatard

Desprez

2014

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

International audienceDistributed computing infrastructures are commonly used for scientific computing, and science gateways provide complete middleware stacks to allow their transparent exploitation by end users. However, administrating such systems manually is time consuming and sub-optimal because of the complexity of the execution conditions. Algorithms and frameworks aiming at automating system administration must deal with online and non-clairvoyant conditions, where most parameters are unknown and evolve over time. We consider the problem of controlling task granularity and fairness among scientific workflows executed in these conditions. We present two self-managing loops monitoring the fineness, coarseness, and fairness of workflow executions, comparing these metrics with thresholds extracted from knowledge acquired in previous executions and planning appropriate actions to maintain these metrics to appropriate ranges. Experiments on the European Grid Infrastructure show that our task granularity control can speed up executions up to a factor of 2 and that our fairness control reduces slowdown variability by 3-7 compared with first-come, first-served. We also study the interaction between granularity control and fairness control: our experiments demonstrate that controlling task granularity degrades fairness but that our fairness control algorithm can compensate this degradation

show abstract

On-Line, Non-clairvoyant Optimization of Workflow Activity Granularity on Grids

Cited by 7 publications

References 13 publications

Elastic Scheduling of Scientific Workflows under Deadline Constraints in Cloud Computing Environments

Elastic Scheduling of Scientific Workflows under Deadline Constraints in Cloud Computing Environments

Self-Management of Operational Issues for Grid Computing

Controlling fairness and task granularity in distributed, online, non‐clairvoyant workflow executions

Contact Info

Product

Resources

About