Developing accurate and scalable simulators of production workflow management systems with WRENCH

Casanova, Henri; Silva, Rafael Ferreira da; Tanaka, Ryan; Pandey, Suraj; Jethwani, Gautam; Koch, William F.; Albrecht, Spencer; Oeth, James; Suter, Frédéric

doi:10.1016/j.future.2020.05.030

Cited by 47 publications

(23 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…the WRENCH framework [31], [32]) that support the WfFormat workflow instance format. In other words, these simulators take as input workflow instances in this format (either from actual workflow executions or synthetically generated) and simulate their executions.…”

Section: Wfsimmentioning

confidence: 99%

“…We use simulators of two state-of-the-art WMSs, Pegasus [3] and Makeflow [28], as a case study for evaluation and validation purposes. These simulators are described in [32] (note that the Makeflow simulator is really a simulator of WorkQueue, an execution engine used by Makeflow). Both Pegasus and Makeflow are being used in production to execute workflows for dozens of high-profile applications in a wide range of scientific domains and on a wide range of platforms.…”

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

“…We used both systems to execute workflows on a cloud environment for the purpose of collecting execution logs for building real workflow instances, as described in Section III. The simulators are built using WRENCH [31], [32], a framework for implementing simulators of WMSs that are accurate and can run scalably on a single computer, while requiring minimal software development effort. The work in [32] demonstrates that WRENCH achieves these objectives, and provides high simulation accuracy for workflow executions using both Pegagus and Makeflow.…”

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

“…In [32], [12], we have already demonstrated that the simulation framework used in our previous set of tools [10] suffers from significant discrepancies from actual executions. These discrepancies mostly stem from the use of a simplistic network simulation model, and from the simulator not capturing relevant details of the system, and thus of the workflow execution.…”

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

See 3 more Smart Citations

WfCommons: A Framework for Enabling Scientific Workflow Research and Development

Coleman¹,

Casanova²,

Pottier³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Scientific workflows are a cornerstone of modern scientific computing. They are used to describe complex computational applications that require efficient and robust management of large volumes of data, which are typically stored/processed on heterogeneous, distributed resources. The workflow research and development community has employed a number of methods for the quantitative evaluation of existing and novel workflow algorithms and systems. In particular, a common approach is to simulate workflow executions. In previous works, we have presented a collection of tools that have been adopted by the community for conducting workflow research. Despite their popularity, they suffer from several shortcomings that prevent easy adoption, maintenance, and consistency with the evolving structures and computational requirements of production workflows. In this work, we present WfCommons, a framework that provides a collection of tools for analyzing workflow executions, for producing generators of synthetic workflows, and for simulating workflow executions. We demonstrate the realism of the generated synthetic workflows by comparing their simulated executions to real workflow executions. We also contrast these results with results obtained when using the previously available collection of tools. We find that the workflow generators that are automatically constructed by our framework not only generate representative same-scale workflows (i.e., with structures and task characteristics distributions that resemble those observed in realworld workflows), but also do so at scales larger than that of available real-world workflows. Finally, we conduct a case study to demonstrate the usefulness of our framework for estimating the energy consumption of large-scale workflow executions.

show abstract

Section: Wfsimmentioning

confidence: 99%

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

Section: Evaluating the Accuracy Of Synthetic Workflow Instancesmentioning

confidence: 99%

See 2 more Smart Citations

WfCommons: A Framework for Enabling Scientific Workflow Research and Development

Coleman¹,

Casanova²,

Pottier³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The simulator is built using WRENCH [21], a framework for implementing simulators of WMSs that are accurate and can run scalably on a single computer. In [22], it was demonstrated that WRENCH provides high simulation accuracy for workflow executions using Pegasus. To ensure accurate and coherent comparisons, all simulation results in this section are obtained for the same simulated platform specification as that of the real-world platforms that was used to obtain the real workflow instances (based on execution logs): 4 compute nodes each with 48 processors on the Chameleon testbed [19].…”

Section: Evaluating the Accuracy Of Synthetic Workflowsmentioning

confidence: 99%

WfChef: Automated Generation of Accurate Scientific Workflow Generators

Coleman¹,

Casanova²,

Silva³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Scientific workflow applications have become mainstream and their automated and efficient execution on largescale compute platforms is the object of extensive research and development. For these efforts to be successful, a solid experimental methodology is needed to evaluate workflow algorithms and systems. A foundation for this methodology is the availability of realistic workflow instances. Dozens of workflow instances for a few scientific applications are available in public repositories. While these are invaluable, they are limited: workflow instances are not available for all application scales of interest. To address this limitation, previous work has developed generators of synthetic, but representative, workflow instances of arbitrary scales. These generators are popular, but implementing them is a manual, labor-intensive process that requires expert application knowledge. As a result, these generators only target a handful of applications, even though hundreds of applications use workflows in production.In this work, we present WfChef, a framework that fully automates the process of constructing a synthetic workflow generator for any scientific application. Based on an input set of workflow instances, WfChef automatically produces a synthetic workflow generator. We define and evaluate several metrics for quantifying the realism of the generated workflows. Using these metrics, we compare the realism of the workflows generated by WfChef generators to that of the workflows generated by the previously available, hand-crafted generators. We find that the WfChef generators not only require zero development effort (because it is automatically produced), but also generate workflows that are more realistic than those generated by hand-crafted generators.

show abstract

Data‐aware and simulation‐driven planning of scientific workflows on IaaS clouds

N'Takpé

Gnimassoun

Oumtanaga

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

The promise of an easy access to a virtually unlimited number of resources makes Infrastructure as a Service Clouds a good candidate for the execution of data-intensive workflow applications composed of hundreds of computational tasks. Thanks to a careful execution planning, workflow management systems can build a tailored compute infrastructure by combining a set of virtual machine instances. However, these applications usually rely on files to handle dependencies between tasks. A storage space shared by all virtual machines may become a bottleneck and badly impact the application execution time. In this article, we propose an original data-aware planning algorithm that leverages two characteristics of a family of virtual machines instances, that is, a large number of cores and a dedicated storage space on fast SSD drives, to improve data locality, hence reducing the amount of data transfers over the network during the execution of a workflow. We also propose a simulation-driven approach to solve a cost-performance optimization problem and correctly dimension the virtual infrastructure onto which execute a given workflow. Experiments conducted with real application workflows show the benefits of the presented algorithms. The data-aware planning leads to a clear reduction of both execution time and volume of data transferred over the network while the simulation-driven approach allows us to dimension the infrastructure in a reasonable time. K E Y W O R D Sdata-intensive workflows, IaaS cloud, makespan reduction, workflow scheduling INTRODUCTIONScientific workflows constitute an appealing approach to express the complex orchestration of interdependent computations and have become mainstream in many scientific domains. 1 They usually allow users to describe the different steps needed to go from a typically vast amount of data generated by a scientific experiment to the production of an original scientific result. The execution of such data-intensive applications made of hundreds of computational tasks on large scale distributed infrastructures is usually handled by a workflow management system (WMS). [2][3][4] Tasks such as resource selection, data management, or computation scheduling are delegated to the WMS, hence hiding the complexity of these operations to the end user.Commodity clusters and computing grids have long been the infrastructures of choice to execute scientific workflows. The institution of the owner of the workflow generally hosts and manages the former, hence easing the access to resources, while the latter allowed scientists to run their workflows at an unprecedented scale by aggregating resources from multiple institutions. With the support of major companies such as Amazon, 5Google, 6 or Microsoft, 7 Infrastructure as a Service (IaaS) clouds have become serious contenders to clusters and grids. Indeed, IaaS clouds combine their respective advantages by providing an easy access to a virtually unlimited amount of resources. Thanks to a careful planning of a workflow

show abstract

Developing accurate and scalable simulators of production workflow management systems with WRENCH

Cited by 47 publications

References 57 publications

WfCommons: A Framework for Enabling Scientific Workflow Research and Development

WfCommons: A Framework for Enabling Scientific Workflow Research and Development

WfChef: Automated Generation of Accurate Scientific Workflow Generators

Data‐aware and simulation‐driven planning of scientific workflows on IaaS clouds

Contact Info

Product

Resources

About