Common motifs in scientific workflows: An empirical analysis

Garijo, Daniel; Alper, Pinar; Belhajjame, Khalid; Corcho, Óscar; Gil, Yolanda; Goble, Carole

doi:10.1109/escience.2012.6404427

Cited by 52 publications

(80 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classification models have been developed to detect additional patterns in structure, usage and data [22]. More high-level patterns, associated to specific cases of use (data curation, analysis) have been identified in Taverna and Wings workflows [23]. Complementary to this work, graph-based approaches have been considered for automatically combining several analysis steps to help the workflow design process [24] while workflow summarization strategies have been developed to tackle workflow complexity [14,25].…”

Section: Related Workmentioning

confidence: 99%

Distilling structure in Taverna scientific workflows: a refactoring approach

et al. 2014

Self Cite

View full text Add to dashboard Cite

BackgroundScientific workflows management systems are increasingly used to specify and manage bioinformatics experiments. Their programming model appeals to bioinformaticians, who can use them to easily specify complex data processing pipelines. Such a model is underpinned by a graph structure, where nodes represent bioinformatics tasks and links represent the dataflow. The complexity of such graph structures is increasing over time, with possible impacts on scientific workflows reuse. In this work, we propose effective methods for workflow design, with a focus on the Taverna model. We argue that one of the contributing factors for the difficulties in reuse is the presence of "anti-patterns", a term broadly used in program design, to indicate the use of idiomatic forms that lead to over-complicated design. The main contribution of this work is a method for automatically detecting such anti-patterns, and replacing them with different patterns which result in a reduction in the workflow's overall structural complexity. Rewriting workflows in this way will be beneficial both in terms of user experience (easier design and maintenance), and in terms of operational efficiency (easier to manage, and sometimes to exploit the latent parallelism amongst the tasks).ResultsWe have conducted a thorough study of the workflows structures available in Taverna, with the aim of finding out workflow fragments whose structure could be made simpler without altering the workflow semantics. We provide four contributions. Firstly, we identify a set of anti-patterns that contribute to the structural workflow complexity. Secondly, we design a series of refactoring transformations to replace each anti-pattern by a new semantically-equivalent pattern with less redundancy and simplified structure. Thirdly, we introduce a distilling algorithm that takes in a workflow and produces a distilled semantically-equivalent workflow. Lastly, we provide an implementation of our refactoring approach that we evaluate on both the public Taverna workflows and on a private collection of workflows from the BioVel project.ConclusionWe have designed and implemented an approach to improving workflow structure by way of rewriting preserving workflow semantics. Future work includes considering our refactoring approach during the phase of workflow design and proposing guidelines for designing distilled workflows.

show abstract

Section: Related Workmentioning

confidence: 99%

Distilling structure in Taverna scientific workflows: a refactoring approach

et al. 2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this work we assume that workflows will be reused, based on a previous study in different workflow systems [6]. By measuring the size, distribution and number of LONI Pipeline groupings, we can assess how groupings and templates are reused, and which design principles are followed by workflow designers (e.g., small groupings for simplicity, big groupings that can be pasted into other workflows, etc.…”

Section: Goalsmentioning

confidence: 99%

“…Most of the workflows have been made public for others to reuse 6 , and range from neuroimaging to genomics. Some of the workflows were developed as early as 2007, and many of them are being used in different institutions.…”

Section: ) User Corpus 2 (Wc2)mentioning

confidence: 99%

See 1 more Smart Citation

FragFlow Automated Fragment Detection in Scientific Workflows

Garijo

Corcho

Gil

et al. 2014

2014 IEEE 10th International Conference on E-Science

Self Cite

View full text Add to dashboard Cite

Abstract-Scientific workflows provide the means to define, execute and reproduce computational experiments. However, reusing existing workflows still poses challenges for workflow designers. Workflows are often too large and too specific to reuse in their entirety, so reuse is more likely to happen for fragments of workflows. These fragments may be identified manually by users as sub-workflows, or detected automatically. In this paper we present the FragFlow approach, which detects workflow fragments automatically by analyzing existing workflow corpora with graph mining algorithms. FragFlow detects the most common workflow fragments, links them to the original workflows and visualizes them. We evaluate our approach by comparing FragFlow results against user-defined sub-workflows from three different corpora of the LONI Pipeline system. Based on this evaluation, we discuss how automated workflow fragment detection could facilitate workflow reuse.

show abstract

“…In [14] graph clustering is used to discover subworkflows from a repository of workflows. Other analysis approaches include [16,32], and [61].…”

Section: Related Workmentioning

confidence: 99%

Scientific workflows for process mining: building blocks, scenarios, and implementation

Bolt

Leoni

Aalst

2015

Int J Softw Tools Technol Transfer

View full text Add to dashboard Cite

Over the past decade process mining has emerged as a new analytical discipline able to answer a variety of questions based on event data. Event logs have a very particular structure; events have timestamps, refer to activities and resources, and need to be correlated to form process instances. Process mining results tend to be very different from classical data mining results, e.g., process discovery may yield end-to-end process models capturing different perspectives rather than decision trees or frequent patterns. A process-mining tool like ProM provides hundreds of different process mining techniques ranging from discovery and conformance checking to filtering and prediction. Typically, a combination of techniques is needed and, for every step, there are different techniques that may be very sensitive to parameter settings. Moreover, event logs may be huge and may need to be decomposed and distributed for analysis. These aspects make it very cumbersome to analyze event logs manually. Process mining should be repeatable and automated. Therefore, we propose a framework to support the analysis of process mining workflows. Existing scientific workflow systems and data mining tools are not tailored towards process mining and the artifacts used for analysis (process models and event logs). This paper structures the basic building blocks needed for process mining and describes various analysis scenarios. Based on these requirements we implemented RapidProM, a tool supporting scientific workflows for process mining. Examples illustrating the different scenarios are provided to show the feasibility of the approach.B Wil M. P. van der Aalst

show abstract

Common motifs in scientific workflows: An empirical analysis

Cited by 52 publications

References 11 publications

Distilling structure in Taverna scientific workflows: a refactoring approach

Distilling structure in Taverna scientific workflows: a refactoring approach

FragFlow Automated Fragment Detection in Scientific Workflows

Scientific workflows for process mining: building blocks, scenarios, and implementation

Contact Info

Product

Resources

About