A Provenance Model for Manually Curated Data

Buneman, Peter; Chapman, Adriane; Cheney, James; Vansummeren, Stijn

doi:10.1007/11890850_17

Cited by 28 publications

(21 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The importance of provenance information in scientific data and workflow management is widely recognized, as witnessed, e.g., by specialized workshops [4,1], research projects [17], and surveys [3,20] dedicated to this topic, and by investigations on foundations of data provenance for queries and transformations [5,9,23]. However, current scientific workflow systems still offer little or no support for queries of interest to the end-users of these systems, e.g., researchers in the life or physical sciences.…”

Section: Introductionmentioning

confidence: 99%

A Model for User-Oriented Data Provenance in Pipelined Scientific Workflows

Bowers

McPhillips

Ludäscher

et al. 2006

Provenance and Annotation of Data

View full text Add to dashboard Cite

Integrated provenance support promises to be a chief advantage of scientific workflow systems over scriptbased alternatives. While it is often recognized that information gathered during scientific workflow execution can be used automatically to increase fault tolerance (via checkpointing) and to optimize performance (by reusing intermediate data products in future runs), it is perhaps more significant that provenance information may also be used by scientists to reproduce results from earlier runs, to explain unexpected results, and to prepare results for publication. Current workflow systems offer little or no direct support for these "scientistoriented" queries of provenance information. Indeed the use of advanced execution models in scientific workflows (e.g., process networks, which exhibit pipeline parallelism over streaming data) and failure to record certain fundamental events such as state resets of processes, can render existing provenance schemas useless for scientific applications of provenance. We develop a simple provenance model that is capable of supporting a wide range of scientific use cases even for complex models of computation such as process networks. Our approach reduces these use cases to database queries over event logs, and is capable of reconstructing complete data and invocation dependency graphs for a workflow run. Abstract. Integrated provenance support promises to be a chief advantage of scientific workflow systems over script-based alternatives. While it is often recognized that information gathered during scientific workflow execution can be used automatically to increase fault tolerance (via checkpointing) and to optimize performance (by reusing intermediate data products in future runs), it is perhaps more significant that provenance information may also be used by scientists to reproduce results from earlier runs, to explain unexpected results, and to prepare results for publication. Current workflow systems offer little or no direct support for these "scientist-oriented" queries of provenance information. Indeed the use of advanced execution models in scientific workflows (e.g., process networks, which exhibit pipeline parallelism over streaming data) and failure to record certain fundamental events such as state resets of processes, can render existing provenance schemas useless for scientific applications of provenance. We develop a simple provenance model that is capable of supporting a wide range of scientific use cases even for complex models of computation such as process networks. Our approach reduces these use cases to database queries over event logs, and is capable of reconstructing complete data and invocation dependency graphs for a workflow run.

show abstract

Section: Introductionmentioning

confidence: 99%

A Model for User-Oriented Data Provenance in Pipelined Scientific Workflows

Bowers

McPhillips

Ludäscher

et al. 2006

Provenance and Annotation of Data

View full text Add to dashboard Cite

show abstract

“…2. Extensibility: Support for the addition of domain specific provenance information (requirement based on the multiplicity of provenance models and applications expressed in [12,13,[15][16][17][18][19][20][21][22], use case III, quality dimensions 2, 3, 5, 6, 11, 12).…”

Section: Requirements For a Provenance Model For The Webmentioning

confidence: 99%

W3P: Building an OPM based provenance model for the Web

Freitas

Knap

Ó’Riain

et al. 2011

Future Generation Computer Systems

View full text Add to dashboard Cite

“…Examples of such systems include operating system [218][159], desktop [265], statistical packages [143,21], databases [59,406,176], data warehouses [104], and workflow systems (Kepler [39,41], Taverna [419], VisTrails [156], VDS [424], Pegasus [278]). Other approaches allow for varying degrees of open-ness: curated databases allow for user-edits [52], PASOA allows for service-oriented architectures [195] where the number of components can dynamically change and is not known at design-time, or some allow for data to be published on the Web, with data and provenance both accessible by simple browser navigation [447]. Others even aim at tracking the provenance of objects in the physical world [231].…”

Section: Assumptionsmentioning

confidence: 99%

“…Buneman et al [51,52] assume that changes to the database can be modeled using transactions comprising simple "copy-paste" updates. They derive a model of provenance that captures such update sequence, which allows them to reconstitute changes to the database.…”

Section: Broadening the Scope Of Provenance Beyondmentioning

confidence: 99%

The Foundations for Provenance on the Web

Moreau

2010

FNT in Web Science

172

View full text Add to dashboard Cite

Provenance, i.e., the origin or source of something, is becoming an important concern, since it offers the means to verify data products, to infer their quality, to analyse the processes that led to them, and to decide whether they can be trusted. For instance, provenance enables the reproducibility of scientific results; provenance is necessary to track attribution and credit in curated databases; and, it is essential for reasoners to make trust judgements about the information they use over the Semantic Web.As the Web allows information sharing, discovery, aggregation, filtering and flow in an unprecedented manner, it also becomes very difficult to identify, reliably, the original source that produced an information item on the Web. Since the emerging use of provenance in niche applications is undoubtedly demonstrating the benefits of provenance, we contend that provenance can and should reliably be tracked and exploited on the Web, and we survey the necessary foundations to achieve such a vision.Using multiple data sources, we have compiled the largest bibliographical database on provenance so far. This large corpus allows us to analyse emerging trends in the research community. Specifically, using the CiteSpace tool, we identify clusters of papers that constitute research fronts, from which we derive characteristics that we use to structure our foundational framework for provenance on the Web. We note that such an endeavour requires a multi-disciplinary approach, since it requires contributions from many computer science sub-disciplines, but also other non-technical fields given the human challenge that is anticipated.To develop our vision, it is necessary to provide a definition of provenance that applies to the Web context. Our conceptual definition of provenance is expressed in terms of processes, and is shown to generalise various definitions of provenance commonly encountered. Furthermore, by bringing realistic distributed systems assumptions, we refine our definition as a query over assertions made by processes.Given that the majority of work on provenance has been undertaken by the database, workflow and e-science communities, we review some of their work, contrasting approaches, and focusing on important topics we believe to be crucial for bringing provenance to the Web, such as abstraction, collections, storage, queries, workflow evolution, semantics and activities involving human interactions.However, provenance approaches developed in the context of databases and workflows essentially deal with closed systems. By that, we mean that workflow or database management systems are in full control of the data they manage, and track their provenance within their own scope, but not beyond. In the context of the Web, a broader approach is required by which chunks of provenance representation can be brought together to describe the provenance of information flowing across multiple systems. This is the specific purpose of the Open Provenance Vision, which is an approach that consists of controlled voc...

show abstract

A Provenance Model for Manually Curated Data

Cited by 28 publications

References 15 publications

A Model for User-Oriented Data Provenance in Pipelined Scientific Workflows

A Model for User-Oriented Data Provenance in Pipelined Scientific Workflows

W3P: Building an OPM based provenance model for the Web

The Foundations for Provenance on the Web

Contact Info

Product

Resources

About