HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows

Baliś, Bartosz

doi:10.1016/j.future.2015.08.015

Cited by 58 publications

(29 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first one is our new suite, designed specifically for this research, based on Serverless Framework. The second one uses our HyperFlow workflow engine . The reason for having two suites was that one of them, namely the HyperFlow, has been already used before to run preliminary experiments on cloud functions, and it allows us to execute workflows that can have many parallel tasks.…”

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

“…For running parallel benchmarking experiments, we adapted HyperFlow workflow engine. HyperFlow was earlier integrated with Google Cloud Functions, and for this work, it was extended to support AWS Lambda.…”

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

“…The second one uses our HyperFlow workflow engine. 1,2 The reason for having two suites was that one of them, namely the HyperFlow, has been already used before to run preliminary experiments on cloud functions, and it allows us to execute workflows that can have many parallel tasks. Therefore, it was natural to use as a driver for our CPU-intensive benchmarks.…”

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

“…We have developed a framework for performance evaluation of cloud functions and applied it to all the major cloud function providers: AWS Lambda, Azure Functions, Google Cloud Functions, and IBM Cloud Functions. Moreover, we used our existing scientific workflow engine HyperFlow, which has been recently extended to support cloud functions, to run parallel workflow benchmarks. We report on the initial results of the performance evaluation, and we discuss the discovered insights into resource allocation policies.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Performance evaluation of heterogeneous cloud functions

Figiela

Gajek

Zima

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Cloud Functions, often called Function‐as‐a‐Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous due to variations in underlying hardware, runtime systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework and one based on HyperFlow. We deployed the CPU‐intensive benchmarks: Mersenne Twister and Linpack. We measured the data transfer times between cloud functions and storage, and we measured the lifetime of the runtime environment. We evaluated all the major cloud function providers: AWS Lambda, Azure Functions, Google Cloud Functions, and IBM Cloud Functions. We made our results available online and continuously updated. We report on the results of the performance evaluation, and we discuss the discovered insights into resource allocation policies.

show abstract

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

Section: Benchmarking Framework For Cloud Functionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Performance evaluation of heterogeneous cloud functions

Figiela

Gajek

Zima

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…3.14 4.14 5.14 6.14 7.14 8.14 9.14 10.14 11.14 12.14 1. 15 In the future, Atmosphere will add support for other workflow management systems such as Kepler [38], Galaxy [24], and HyperFlow [39], while also improving its optimisation mechanisms for heterogeneous clouds. Another interesting area of research is to support container technologies (e.g.…”

Section: Discussionmentioning

confidence: 99%

Support for Taverna workflows in the VPH-Share cloud platform

Kasztelnik

Coto

Bubak

et al. 2017

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

Improving prediction of computational job execution times with machine learning

Balis,

Lelek,

Bodera

et al. 2023

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SummaryPredicting resource consumption and run time of computational workloads is crucial for efficient resource allocation, or cost and energy optimization. In this paper, we evaluate various machine learning techniques to predict the execution time of computational jobs. For experiments we use datasets from two application areas: scientific workflow management and data processing in the ALICE experiment at CERN. We apply a two‐stage prediction method and evaluate its performance. Other evaluated aspects include: (1) comparing performance of global (per‐workflow) versus specialized (per‐job) models; (2) impact of prediction granularity in the first stage of the two‐stage method; (3) using various feature sets, feature selection, and feature importance analysis; (4) applying symbolic regression in addition to classical regressors. Our results provide new valuable insights on using machine learning techniques to predict the runtime behavior of computational jobs.

show abstract

HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflows

Cited by 58 publications

References 18 publications

Performance evaluation of heterogeneous cloud functions

Performance evaluation of heterogeneous cloud functions

Support for Taverna workflows in the VPH-Share cloud platform

Improving prediction of computational job execution times with machine learning

Contact Info

Product

Resources

About