Uncertainty Quantification in Computational Predictive Models for Fluid Dynamics Using a Workflow Management Engine

Guerra, Gabriel M.; Rochinha, Fernando A.; Elias, Renato N.; Oliveira, Daniel de; Ogasawara, Eduardo; Dias, Jonas; Mattoso, Marta; Coutinho, Álvaro L. G. A.

doi:10.1615/int.j.uncertaintyquantification.v2.i1.50

Cited by 12 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Engineering problems may require reliable numerical simulations, which may fail due to the presence of uncertainty in the model data, such as constitutive equation parameters [80]. To cope with uncertainty, reliability methods such as uncertainty quantification (UQ) are used.…”

Section: Workflow Ensemblesmentioning

confidence: 99%

A characterization of workflow management systems for extreme-scale applications

Silva

Filgueira

Pietri

et al. 2017

Future Generation Computer Systems

126

View full text Add to dashboard Cite

Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today's computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.

show abstract

Section: Workflow Ensemblesmentioning

confidence: 99%

A characterization of workflow management systems for extreme-scale applications

Silva

Filgueira

Pietri

et al. 2017

Future Generation Computer Systems

126

View full text Add to dashboard Cite

show abstract

“…We then started working with bigger problems such as Uncertainty Quantification (UQ) [14]. Adaptations of the workflow at runtime in [13] were still hand-made through provenance database queries, but the performance improvements obtained, motivated us to develop specific constructs to support dynamic workflows.…”

Section: Introductionmentioning

confidence: 99%

“…In order to store domain data, our approach requires the workflow to be instrumented so that the execution engine can capture data and store it in the provenance database during runtime. Consequently, users can query real-time provenance [14,24], which is essential to support decision making during the dynamic steering. In [14], for example, a UQ workflow is improved using realtime provenance queries even without dynamic loops support.…”

Section: Introductionmentioning

confidence: 99%

“…Consequently, users can query real-time provenance [14,24], which is essential to support decision making during the dynamic steering. In [14], for example, a UQ workflow is improved using realtime provenance queries even without dynamic loops support. The scientists can monitor several execution parameters, like converged error, average and variance of kinetic energy, by accessing partial workflow result analysis through provenance queries.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data-centric iteration in dynamic workflows

Dias

Guerra

Rochinha

et al. 2015

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

SummaryDynamic workflows are scientific workflows to support computational science simulations, typically using dynamic processes based on runtime scientific data analyses. They require the ability of adapting the workflow, at runtime, based on user input and dynamic steering. Supporting data-centric iteration is an important step towards dynamic workflows because user interaction with workflows is iterative. However, current support for iteration in scientific workflows is static and does not allow for changing data at runtime. In this paper, we propose a solution based on algebraic operators and a dynamic execution model to enable workflow adaptation based on user input and dynamic steering. We introduce the concept of iteration lineage that makes provenance data management consistent with dynamic iterative workflow changes. Lineage enables scientists to interact with workflow data and configuration at runtime through an API that triggers steering. We evaluate our approach using a novel and real large-scale workflow for uncertainty quantification on a 640-core cluster. The results show impressive execution time savings from 2.5 to 24 days, compared to noniterative workflow execution. We verify that the maximum overhead introduced by our iterative model is less than 5% of execution time. Also, our proposed steering algorithms are very efficient and run in less than 1 millisecond, in the worst-case scenario.

show abstract

“…Even though the workflow execution log could be browsed, this is far from provenance data query. Our previous works in supporting workflow scientists from bioinformatics [12] and numerical methods [13] have led us to develop services to query provenance data during the execution [14]. We define this type of provenance as runtime provenance data.…”

Section: Introductionmentioning

confidence: 99%

Using Domain-Specific Data to Enhance Scientific Workflow Steering Queries

Gonçalves

Oliveira

Ocaña

et al. 2012

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. In scientific workflows, provenance data helps scientists in understanding, evaluating and reproducing their results. Provenance data generated at runtime can also support workflow steering mechanisms. Steering facilities for workflows is considered a challenge due to its dynamic demands during execution. To steer, for example, scientists should be able to suspend (or stop) a workflow execution when the approximate solution meets (or deviates) preset criteria. These criteria are commonly evaluated based on provenance data (execution data) and domain-specific data. We claim that the final decision on whether to interfere on the workflow execution may only become feasible when workflows can be steered by scientists using provenance data enriched with domain-specific data. In this paper we propose an approach based on specialized software components, named Data Extractor (DE), to acquire domain-specific data from data files produced during a scientific workflow execution. DE gathers domain-specific data from produced data files and associates it to existing provenance data on the provenance repository. We have evaluated the proposed approach using a real bioinformatics workflow for comparative genomics executed in SciCumulus cloud workflow parallel engine.

show abstract

Uncertainty Quantification in Computational Predictive Models for Fluid Dynamics Using a Workflow Management Engine

Cited by 12 publications

References 36 publications

A characterization of workflow management systems for extreme-scale applications

A characterization of workflow management systems for extreme-scale applications

Data-centric iteration in dynamic workflows

Using Domain-Specific Data to Enhance Scientific Workflow Steering Queries

Contact Info

Product

Resources

About