Dynamic workflow support in gUSE

Abstract

Scientific workflow systems aim to provide user friendly, end-to-end solutions for automating and simplifying computational or data intensive tasks. A number of workflow environments have been developed in recent years to provide support for the specification and execution of scientific workflows. Normal static workflows can poorly cope with the ever changing status of the existing distributed systems. During workflow enactment unforeseen scenarios may arise, which can cause significant delays, failed executions, or improper results. Manual workflow enactment has its obvious limitations, however automatic failover mechanisms require minimum an accurate information set about the workflow tasks and about the status of the underlying processing infrastructure. Dynamism can be defined at different abstraction levels and in different phases of the workflow lifecycle. In this paper we identify the requirements of dynamic workflows in general and provide a thorough survey about gUSE/WS-PGRADE's dynamic workflow handling capabilities.

Achieving dynamic workflow management system by applying provenance based checkpointing method

“…We define global failure cost (4) of a task A i : which is the execution time overhead of the whole workflow, when one failure occurs during Task A i (4) where (5) and (6) are classic formulas that are used in tasks scheduling [12] [11]. Basically the rank() function is the critical path from task A i to the last task, and can be computed recursively backward from the last task.…”

“…During such long intervals it is inevitable to adapt to the dynamically changing environment which can be caused by unwanted input data, crash faults or network problems. In of our earlier works [4] we defined the main requirements of dynamic workflow execution systems as: the ability to react to or to handle unforeseen scenarios raised during the workflow enactment phase, to adapt to new situations, to change the abstract or concrete workflow model or to give faster execution and higher level performance according to the actual environmental conditions and intermediary results. In our other work [2] we have defined the three main areas of dynamism which are optimization of the workflow execution according some criteria, user-steering (user or administrator interaction during execution) [3] and fault tolerance behavior.…”

Usability of Scientific Workflow in Dynamically Changing Environment

“…In one of our earlier work [13] we have summarized the requirements of a dynamic workflow management system regarding three phases (design, instantiation, execution) of the workflow lifecycle. In each phase (which can be interpreted as different abstract level as well) we have differentiated additional levels in order to have a deeper insight about this topic.…”

Minimal sufficient information about the scientific workflows to create reproducible experiment

“…The datasets are created in the different phases of the scientific workflow lifecycle [8], [9] and originate from three different sources. The scientist can give information when to design the abstract model, when to get the results or after the results are published.…”