Automate Scientific Workflow Execution between Local Cluster and Cloud

Abstract

Scientific computational experiments often span multiple computational and analytical steps, and during execution, researchers need to store, access, transfer, and query information. Scientific workflow is a powerful tool to streamline and organize scientific application. Numbers of tools have been developed to help build scientific workflows, they provide mechanisms for creating workflow but lack a native scheduling system for determining where code should be executed. This paper presents Emerald, a system that adds sophisticated computation offloading capabilities to scientific workflows. Emerald automatically offloads computation intensive steps of scientific workflow to the cloud in order to enhance workflow performance. Emerald minimizes the burden on developers to build workflows with computation offloading ability by providing easy-to-use API. Evaluation showed that Emerald can effectively reduce up to 55% of execution time for scientific applications.

Modeling multi-site computation offloading in unreliable cloud environments

“…Researchers in [19,20,21,22] have worked on extending the battery life of the mobile device. In works done in [23,24,25,6] their main goal was reducing the execution time.…”

Building a blockchain-based decentralized ecosystem for cloud and edge computing: an ALLSTAR approach and empirical study

“…For example, the Edge resources are provided by base stations and are owned by the Shanghai Telecom company in Guo et al (2020). By contrast, application operators utilize their own local cluster to work as Edge resources in Qian and Andresen (2016). Teerapittayanon et al (2017) propose to deploy the neural network across multiple Edge devices, such as sensors and cameras, in a distributed manner.…”