Techniques for efficiently querying scientific workflow provenance graphs

Anand, Manish Kumar; Bowers, Shawn; Ludäscher, Bertram

doi:10.1145/1739041.1739078

Cited by 60 publications

(47 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the difficulty to support process analysis based on data provenance is mainly the complexity and variety of the analysis requirement, which proposes a serious description problem for data provenance querying. Then the issue of data provenance querying is just addressed in a application-independent way to in recent years, the query languages of data provenance are proposed to resolve the problem, like OPQL [6], VQuel [7], ProQL [8], QLP [9], etc. These query languages are based on formal models for data provenance, which cause the difficulty in writing query by these languages.…”

Section: Related Workmentioning

confidence: 99%

A Query Language of Data Provenance Based on Dependency View for Process Analysis

Sun¹,

Gao

Du³

et al. 2016

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Abstract-For the scale of data in process keep increasing, data provenance also becomes large and constantly growing, which brings challenges to the efficiency of provenance tracking in process analysis. This paper proposes a kind of dependency view to extract a global data provenance description of the data process instance, and then defines a contextual query language based on dependency view to implement an efficient provenance query mechanism for process analysis. The elements of the language are based on a set of dependency view query operations, which can decrease the steps of provenance tracking based on the elements of data provenance and support the descriptive power of the language for complex provenance tracking. Experimental results show that complex provenance tracking by the language is efficient and ease to use.

show abstract

Section: Related Workmentioning

confidence: 99%

A Query Language of Data Provenance Based on Dependency View for Process Analysis

Sun¹,

Gao

Du³

et al. 2016

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…Provenance management has been acknowledged as a critical functionality for any SWFMS [12,[17][18][19][20][21]; see [22,23] for surveys. Provenance supports the reproducibility of scientific results.…”

Section: Provenance Modelsmentioning

confidence: 99%

Collaborative scientific workflows supporting collaborative science

Zhang

2011

IJBPIM

View full text Add to dashboard Cite

Collaboration has become a dominant feature of modern science. Many scientific problems are beyond the realm of individual discipline or scientist to solve and hence require collaborative efforts. Meanwhile, today's science becomes increasingly more dataintensive, resulting in a rapid transition from computational science to e-Science (or digital science). Recently, scientific workflows have emerged for scientists to integrate distributed computations, datasets, and analysis tools to enable and accelerate scientific discovery. The convergence of the above two trends naturally leads to the concept of collaborative scientific workflows. This paper presents a disciplinary definition of this term, discusses the opportunities, requirements, and challenges of collaborative scientific workflows for the enablement of scientific collaboration, and concludes with our ongoing work in this direction.

show abstract

“…Such queries are often complex to express and expensive to evaluate [12,13,14,15]. QLP provides a simple, declarative, path-based language (similar, e.g., to XPath) for expressing such queries, and optimization techniques have been developed that make answering QLP queries over large provenance repositories feasible [16]. QLP queries work over sets of lineage edges, e.g., represented by the DerivedFrom relation.…”

Section: Expressing Collaborative Queriesmentioning

confidence: 99%

“…We write v(p) to denote the set of lineage edges of the form d 2 , r, d 1 ∈ L returned after evaluating a QLP path query p over a set of lineage edges L [16]. We directly use this evaluation to define the DATA-DEP function as follows.…”

Section: Filtering Dependency Views Using Qlpmentioning

confidence: 99%

“…Approaches for querying provenance are largely based on physical data representations [14], e.g., relational, XML, or RDF schemas, where users express provenance queries through corresponding query languages, i.e., SQL, XQuery, or SPARQL. Provenance queries often require computing transitive closures over dependency relations, and expressing such queries using standard approaches is typically done using recursion or stored procedures [15,16,27]. Expressing such queries is both cumbersome and error-prone, and requires considerable user expertise.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Collaborative Studies through Interoperable Workflow Provenance

Altıntaş

Anand

Crawl

et al. 2010

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. The provenance of a data product contains information about how the product was derived, and is crucial for enabling scientists to easily understand, reproduce, and verify scientific results. Currently, most provenance models are designed to capture the provenance related to a single run, and mostly executed by a single user. However, a scientific discovery is often the result of methodical execution of many scientific workflows with many datasets produced at different times by one or more users. Further, to promote and facilitate exchange of information between multiple workflow systems supporting provenance, the Open Provenance Model (OPM) has been proposed by the scientific workflow community. In this paper, we describe a new query model that captures implicit user collaborations. We show how this model maps to OPM and helps to answer collaborative queries, e.g., identifying combined workflows and contributions of users collaborating on a project based on the records of previous workflow executions. We also adopt and extend the high-level Query Language for Provenance (QLP) with additional constructs, and show how these extensions allow non-expert users to express collaborative provenance queries against this model easily and concisely. Furthermore, we adopt the Provenance Challenge 3 (PC3) workflows as a collaborative and interoperable usecase scenario, where different stages of the workflow are executed in three different workflow environmentsKepler, Taverna, and WSVLAM. Through this usecase, we demonstrate how we can establish and understand collaborative studies through interoperable workflow provenance.

show abstract

Techniques for efficiently querying scientific workflow provenance graphs

Cited by 60 publications

References 22 publications

A Query Language of Data Provenance Based on Dependency View for Process Analysis

A Query Language of Data Provenance Based on Dependency View for Process Analysis

Collaborative scientific workflows supporting collaborative science

Understanding Collaborative Studies through Interoperable Workflow Provenance

Contact Info

Product

Resources

About