Fostering Scientific Workflow Preservation through Discovery of Substitute Services

Abstract

Abstract-Scientific workflows are increasingly gaining momentum as the new paradigm for modeling and enacting scientific experiments. The value of a workflow specification does not end once it is enacted. Indeed, workflow specifications encapsulate knowledge that documents scientific experiments, and are, therefore, worth preserving.Our experience suggests that workflow preservation is frequently hampered by the volatility of the constituent service operations when these operations are supplied by third-party providers. To deal with this issue, we propose a heuristic for locating substitutes that are able to replace unavailable service operations within workflows. The proposed method uses the data links connecting inputs and outputs of service operations in existing workflow specifications to locate operations with parameters compatible with those of the missing operations. Furthermore, it exploits provenance traces collected from past executions of workflows to ensure that candidate substitutes perform tasks similar to those of the missing operations. The effectiveness of the proposed method has been empirically assessed.

Using a suite of ontologies for preserving workflow-centric research objects

“…Provenance traces of executions allow users to track how results were produced by the workflow, and repair broken workflows [6]. Past studies have shown the usefulness of provenance information in supporting workflow reproducibility [6,[16][17][18]. The issues described in Section 1 could all benefit from the availability of detailed provenance information: issue 1 by replaying how the workflow functions [16] using the complete trace of all the computational tasks taking place in the workflow; issue 2, by finding example inputs data used by the workflow; issue 3, by retrieving the intermediate results produced in the original runs to resume workflow runs from the failure point; and finally issue 4, by retrieving information about the original computational environment, like the OS, library dependencies as well as their versions. Extensive provenance tracking is the focus of many reproducibility efforts, like VisTrails [16] and CDE [19], etc.…”

“…R 2 Workflows should be preserved together with provenance traces of their data results. Provenance traces of executions allow users to track how results were produced by the workflow, and repair broken workflows [6]. Past studies have shown the usefulness of provenance information in supporting workflow reproducibility [6,[16][17][18].…”

“…The workflow description vocabulary (wfdesc) 6 is used to describe the workflow specifications included in a Workflow Research Object. The features of the ontology were established by an examination of the core and overlapping concepts used in 3 major data driven workflow systems, Taverna [37], Wings [38] and Galaxy [39].…”

“…Issue 3 is instead a consequence of using 3rd party services as part of a workflow, and is a relevant issue in workflow decay [4]. Whilst the loss of 3rd party services is out of the control of original authors, there are a number of approaches to remedy this type of workflow decay by making use of metadata -such as additional semantic descriptions about the services used [5], or provenance information [6][7][8] -all of which can be either provided by the author of the workflow or automatically tracked and computed.…”

Producing Data, Producing Software: Developing a Radio Astronomy Research Infrastructure

“…The eScience community has a well-established history of designing and developing software pipelines for scientific data production, processing, and analysis [1,4,6]. These pipelines often enable the sharing of data.…”