Autonomic Activities in the Execution of Scientific Workflows: Evaluation of the AWARD Framework

Abstract

Abstract-Workflows have been successfully applied to express the decomposition of complex scientific applications. However the existing tools still lack adequate support to important aspects namely, decoupling the enactment engine from tasks specification, decentralizing the control of workflow activities allowing their tasks to run in distributed infrastructures, and supporting dynamic workflow reconfigurations. We present the AWARD (Autonomic Workflow Activities Reconfigurable and Dynamic) model of computation, based on Process Networks, where the workflow activities (AWA) are autonomic processes with independent control that can run in parallel on distributed infrastructures. Each AWA executes a task developed as a Java class with a generic interface allowing end-users to code their applications without low-level details. The data-driven coordination of AWA interactions is based on a shared tuple space that also enables dynamic workflow reconfiguration. For evaluation we describe experimental results of AWARD workflow executions in several application scenarios, mapped to the Amazon (Elastic Computing EC2) Cloud.

Raw data queries during data-intensive parallel workflow execution

“…As shown in [16], the AWARD framework [26] has a dataflow approach based on the physical level. It captures tuples (set of parameter values) at runtime and manages data propagation through simulation programs according to the dependency between existing data.…”

Dynamic steering of HPC scientific workflows: A survey

“…In [29,37] domain data to be monitored at runtime is configured by the user, so that it can be extracted during execution and available for queries during runtime. Assunção et al [50] keep intermediate domain data produced by the workflow in a shared tuple space, allowing scientists to interact with them at runtime. This integration between different levels of monitoring information may require special organization in the storage of monitoring data.…”

“…This integration between different levels of monitoring information may require special organization in the storage of monitoring data. For instance, most approaches use relational databases [25], while [50] uses a shared tuple space to store different levels of monitoring and provenance. Relational queries can obtain average execution time of tasks at runtime, which is very helpful to identify an outlier execution.…”

“…Dynamic data collection definition by the user is only supported by the Aware approach of Assunção et al [50]. This gives a lot of flexibility to avoid information overloading or lack of specific data for monitoring.…”

“…Several SWfMS present an adaptive management using autonomic agents [45,50,65]. These solutions schedule tasks to execute remotely using available services in grids and clouds.…”