Exploiting execution provenance to explain difference between two data-intensive computations

Thavasimani, Priyaa; Cała, Jacek; Missier, Paolo

doi:10.1109/escience.2018.00109

Cited by 3 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The interest in workflow provenance management has increased in the recent years, driven by a major effort by the provenance community, 31,35,[50][51][52][53][54][55][56][57][58][59][60][61][62] particularly to explore possibilities of optimizing workflows with the data captured by provenance tools and as a response to the urgent need for reproducible science, which is critical in scientific ML. 63 To exemplify, Thavasimani et al 14 investigate provenance traces recorded during workflow executions to observe differences in results with minor workflow configuration differences.…”

Section: Related Workmentioning

confidence: 99%

Workflow provenance in the lifecycle of scientific machine learning

Souza¹,

Azevedo²,

Lourenço³

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Machine learning (ML) has already fundamentally changed several businesses. More recently, it has also been profoundly impacting the computational science and engineering domains, like geoscience, climate science, and health science. In these domains, users need to perform comprehensive data analyses combining scientific data and ML models to provide for critical requirements, such as reproducibility, model explainability, and experiment data understanding. However, scientific ML is multidisciplinary, heterogeneous, and affected by the physical constraints of the domain, making such analyses even more challenging. In this work, we leverage workflow provenance techniques to build a holistic view to support the lifecycle of scientific ML. We contribute with (i) characterization of the lifecycle and taxonomy for data analyses; (ii) design decisions to build this view, with a W3C PROV compliant data representation and a reference system architecture; and (iii) lessons learned after an evaluation in an Oil & Gas case using an HPC cluster with 393 nodes and 946 GPUs. The experiments show that the decisions enable queries that integrate domain semantics with ML models while keeping low overhead (<1%), high scalability, and an order of magnitude of query acceleration under certain workloads against without our representation.

show abstract

Section: Related Workmentioning

confidence: 99%

Workflow provenance in the lifecycle of scientific machine learning

Souza¹,

Azevedo²,

Lourenço³

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…The interest in workflow provenance management has increased in the recent years, driven by a major effort by the provenance community [46], [47], [48], [49], [50], [51], [31], [52], [53], [35], [54], [55], [56], [57], [58] , particularly to explore possibilities of optimizing workflows with the data captured by provenance tools and as a response to the urgent need for reproducible science, which is critical in scientific ML [59]. To exemplify, Thavasimani et al, [14] investigate provenance traces recorded during workflow executions to observe differences in results with minor workflow configuration differences.…”

Section: Related Workmentioning

confidence: 99%

Workflow Provenance in the Lifecycle of Scientific Machine Learning

Souza,

Azevedo,

Lourenço

et al. 2020

Preprint

View full text Add to dashboard Cite

Machine Learning (ML) has already fundamentally changed several businesses. More recently, it has also been profoundly impacting the computational science and engineering domains, like geoscience, climate science, and health science. In these domains, users need to perform comprehensive data analyses combining scientific data and ML models to provide for critical requirements, such as reproducibility, model explainability, and experiment data understanding. However, scientific ML is multidisciplinary, heterogeneous, and affected by the physical constraints of the domain, making such analyses even more challenging. In this work, we leverage workflow provenance techniques to build a holistic view to support the lifecycle of scientific ML. We contribute with (i) characterization of the lifecycle and taxonomy for data analyses; (ii) design principles to build this view, with a W3C PROV compliant data representation and a reference system architecture; and (iii) lessons learned after an evaluation in an Oil & Gas case using an HPC cluster with 393 nodes and 946 GPUs. The experiments show that the principles enable queries that integrate domain semantics with ML models while keeping low overhead (<1%), high scalability, and an order of magnitude of query acceleration under certain workloads against without our representation.

show abstract

“…Here we extend our previous study to cover the general case where E is obtained from E through a sequence of edits, and as a consequence, the traces collected at runtime are non-isomorphic and reflect structural differences introduced through these edits. The abstract idea of comparing two non-isomorphic graphs is proposed in our earlier work [16].…”

Section: B Contributionsmentioning

confidence: 99%

Why-Diff: Exploiting Provenance to Understand Outcome Differences From Non-Identical Reproduced Workflows

2019

Self Cite

View full text Add to dashboard Cite

Data analytics processes such as scientific workflows tend to be executed repeatedly, with varying dependencies and input datasets. The case has been made in the past for tracking the provenance of the final information products through the workflow steps, to enable their reproducibility. In this paper, we explore the hypothesis that provenance traces recorded during execution are also instrumental to answering questions about the observed differences between sets of results obtained from similar but not identical workflow configurations. Such differences in configurations may be introduced deliberately, i.e., to explore process variations or accidentally, typically as the result of porting efforts or of changes in the computing environment. Using a commonly used workflow programming model as a reference, we consider both structural variations in the workflows as well as variations within their individual components. Our why-diff algorithm compares the graph representations of two provenance traces derived from two workflow variations. It produces a delta graph that can be used to produce human-readable explanations of the impact of workflow differences on observed output differences. We report with Neo4j graph database. Further, we report explanations of the difference between workflow results using a suite of synthetic workflows as well as real-world workflows. INDEX TERMS Why-diff, provenance, reproducibility, big data, eScience Central, workflow.

show abstract

Exploiting execution provenance to explain difference between two data-intensive computations

Cited by 3 publications

References 4 publications

Workflow provenance in the lifecycle of scientific machine learning

Workflow provenance in the lifecycle of scientific machine learning

Workflow Provenance in the Lifecycle of Scientific Machine Learning

Why-Diff: Exploiting Provenance to Understand Outcome Differences From Non-Identical Reproduced Workflows

Contact Info

Product

Resources

About