Benchmarking Heterogeneous Cloud Functions

Malawski, Maciej; Figiela, Kamil; Gajek, Adam; Zima, Adam

doi:10.1007/978-3-319-75178-8_34

Cited by 39 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper is an extension of our earlier conference publication, in which we presented the preliminary results of our benchmarks. In this paper, we extend this publication by: (i) presenting the results of long running analysis of CPU performance for the period of April – September 2017 – March 2018, (ii) adding new measurements of data transfers and instance lifetime, and (iii) presenting the performance‐cost analysis.…”

Section: Introductionmentioning

confidence: 99%

Performance evaluation of heterogeneous cloud functions

Figiela

Gajek

Zima

et al. 2018

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Cloud Functions, often called Function‐as‐a‐Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous due to variations in underlying hardware, runtime systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework and one based on HyperFlow. We deployed the CPU‐intensive benchmarks: Mersenne Twister and Linpack. We measured the data transfer times between cloud functions and storage, and we measured the lifetime of the runtime environment. We evaluated all the major cloud function providers: AWS Lambda, Azure Functions, Google Cloud Functions, and IBM Cloud Functions. We made our results available online and continuously updated. We report on the results of the performance evaluation, and we discuss the discovered insights into resource allocation policies.

show abstract

Section: Introductionmentioning

confidence: 99%

Performance evaluation of heterogeneous cloud functions

Figiela

Gajek

Zima

et al. 2018

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…To ease the simulation of workflow computing, several researchers have contributed to develop simulators specifically for workflow simulation. 14,15 WorkflowSim 14 is an open-source workflow simulator based on CloudSim. It supports workflow simulation by adding components such as Workflow Mapper, Workflow Engine, and Workflow Scheduler.…”

Section: Related Workmentioning

confidence: 99%

“…Therefore, both WorkflowSim 14 and CloudWorkflowSimulator 15 provide several visualization functions for workflow execution schedule. In CloudWorkflowSimulator, 15 users can even adjust the resolution of the visualized schedule or focus on part of the schedule by the crop function.…”

Section: Related Workmentioning

confidence: 99%

“…It has been developed aiming for providing a common performance simulation infrastructure supporting research work on different types of workflow scheduling problems. In contrast to most existing stand-alone workflow scheduling simulators, 14,15 Pewss was developed as a cloud service based on the SaaS model and adopts a multiuser Web-based client/server architecture. It can increase efficiency, reduce complexity, and avoid human negligence during conducting of simulation experiments for evaluation of the performance of workflow scheduling algorithms.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Pewss: A platform of extensible workflow simulation service for workflow scheduling research

et al. 2017

View full text Add to dashboard Cite

This paper presents a Platform of Extensible Workflow Simulation Service (Pewss), which we have developed to provide a cloud service for aiding research work in workflow scheduling. The simulation has been a major tool for performance evaluation and comparison in workflow scheduling research. However, researchers usually have to develop their own simulation programs with limited functionality, simply outputting summarized performance results. Pewss has been developed for easing and improving current practices in conducting performance simulations during studying of existing workflow scheduling algorithms or designing of new scheduling algorithms. Pewss has been designed based on the Software as a Service (SaaS) model, adopting a multiuser Web-based client/server architecture. Conducting simulation experiments on Pewss, researchers simply have to implement the scheduling algorithm under study instead of a whole simulation environment, allowing them to focus on their research work without spending unnecessary efforts on the simulation implementation details. Pewss provides the visualization of a workflow execution schedule based on simulation results, offering a convenient way for researchers to gain an insight into the effectiveness, characteristics, and performance bottleneck of scheduling algorithms. As a multiuser environment, Pewss also provides functionality for researchers to facilitate comparative performance analysis and collaborative research works effectively. Pewss has been used in our research work on task-parallel workflow scheduling and has been planned to be extended to support other types of workflow scheduling research problems, eg, mixed-parallel workflows.

show abstract

“…Under the hood, all platform are heterogeneous in nature, because they use different hardware and software components, different runtime systems and programming languages, routing gateways, resource management tools and monitoring systems. As an example, Malawski et al (2017) points out that most platforms use Linux, but Azure functions run on Windows. With the rising popularity of FaaS, an increasing need to benchmark different aspects of FaaS platforms has been recognized by research and industry (Kuhlenkamp and Werner, 2018).…”

Section: Introductionmentioning

confidence: 99%

Function-as-a-Service Benchmarking Framework

Pellegrini¹,

Ivkic

Tauber

2019

Proceedings of the 9th International Conference on Cloud Computing and Services Science

View full text Add to dashboard Cite

Cloud Service Providers deliver their products in form of "as-a-Service", which are typically categorized by the level of abstraction. This approach hides the implementation details and shows only functionality to the user. However, the problem is that it is hard to measure the performance of Cloud services, because they behave like black boxes. Especially with Function-as-a-Service it is even more difficult because it completely hides server and infrastructure management from users by design. Cloud Service Prodivers usually restrict the maximum size of code, memory and runtime of Cloud Functions. Nevertheless, users need clarification if more ressources are needed to deliver services in high quality. In this regard, we present the architectural design of a new Function-as-a-Service benchmarking tool, which allows users to evaluate the performance of Cloud Functions. Furthermore, the capabilities of the framework are tested on an isolated platform with a specific workload. The results show that users are able to get insights into Function-as-a-Service environments. This, in turn, allows users to identify factors which may slow down or speed up the performance of Cloud Functions.

show abstract

Benchmarking Heterogeneous Cloud Functions

Cited by 39 publications

References 9 publications

Performance evaluation of heterogeneous cloud functions

Performance evaluation of heterogeneous cloud functions

Pewss: A platform of extensible workflow simulation service for workflow scheduling research

Function-as-a-Service Benchmarking Framework

Contact Info

Product

Resources

About