Using Provenance to Support Real-Time Collaborative Design of Workflows

Ellkvist, Tommy; Koop, David; Anderson, Erik; Freire, Juliana; Silva, Cláudio

doi:10.1007/978-3-540-89965-5_27

Cited by 25 publications

(14 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…VIKING is an ESO public survey in Y,Z,J,H,K bands. The reduced images were provided by Cambridge Astronomy Survey Unit 7 and catalogues were extracted in Astro-WISE. This is seven square degree area which is covered in ten filters (u,g,r,i,z-for CFHT, Z,Y,J,H,K-for VIKING).…”

Section: Detailed Use-casementioning

confidence: 99%

“…The use of lineage we describe in this paper is analogous to how other authors have used lineage to solve some use-cases. For example in [2,23] and [18] data lineage has been used for results verification and to prove robustness of methods, in Kepler [1] data lineage has been used to enable smart "reruns" and process simplification of previously executed workflows and in [22] and [7], data lineage has been used for interactive design of workflows.…”

Section: Related Workmentioning

confidence: 99%

“…The use of data lineage has also gained significant attention [7,25]. Several workflow management systems do exploit data lineage for different purposes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sub-image data processing in Astro-WISE

et al. 2012

View full text Add to dashboard Cite

Most often, astronomers are interested in a source (e.g., moving, variable, or extreme in some colour index) that lies on a few pixels of an image. However, the classical approach in astronomical data processing is the processing of the entire image or set of images even when the sole source of interest may exist on only a few pixels of one or a few images. This is because pipelines have been written and designed for instruments with fixed detector properties (e.g., image size, calibration frames, overscan regions, etc.). Furthermore, all metadata and processing parameters are based on an instrument or a detector. Accordingly, out of many thousands of images for a survey, this can lead to unnecessary processing of data that is both timeconsuming and wasteful. We describe the architecture and an implementation of sub-image processing in Astro-WISE. The architecture enables a user to select, retrieve and process only the relevant pixels in an image where the source exists. We show that lineage data collected during the processing and analysis of datasets can be reused to perform selective reprocessing (at subimage level) on datasets while the remainder of the dataset is untouched, a difficult process to automate without lineage.

show abstract

Section: Detailed Use-casementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Sub-image data processing in Astro-WISE

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Several stand-alone provenance systems have also been developed [30][31][32][33][34], including the PReServ system developed under the Provenance Aware Service Oriented Architecture (PASOA) project [32] and the Karma system [33]. Both systems support Web services interfaces.…”

Section: Provenance Modelsmentioning

confidence: 99%

Collaborative scientific workflows supporting collaborative science

Zhang

2011

IJBPIM

View full text Add to dashboard Cite

Collaboration has become a dominant feature of modern science. Many scientific problems are beyond the realm of individual discipline or scientist to solve and hence require collaborative efforts. Meanwhile, today's science becomes increasingly more dataintensive, resulting in a rapid transition from computational science to e-Science (or digital science). Recently, scientific workflows have emerged for scientists to integrate distributed computations, datasets, and analysis tools to enable and accelerate scientific discovery. The convergence of the above two trends naturally leads to the concept of collaborative scientific workflows. This paper presents a disciplinary definition of this term, discusses the opportunities, requirements, and challenges of collaborative scientific workflows for the enablement of scientific collaboration, and concludes with our ongoing work in this direction.

show abstract

“…The use of provenance has also gained significant attention [4]. Several workflow management systems (e.g [2,3,7,9]) do exploit provenance information for different purposes. To the best of our knowledge, this is the first work that leverages lineage information to support sub-image processing to simplify and automate the reprocessing of objects.…”

Section: Introductionmentioning

confidence: 99%