Investigating Performance and Cost in Function-as-a-Service Platforms

Bortolini, Diogo; Obelheiro, Rafael R.

doi:10.1007/978-3-030-33509-0_16

Cited by 11 publications

(19 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AWS Lambda claims that it "allocates CPU power in proportion to the amount of memory configured" 18 . One study confirmed this statement in an ideal world with CPU intensive functions [47] which corresponds to our findings [9]. Resource scaling and runtime prediction is not that deterministic on the other platforms namely GCF and IBM OpenWhisk.…”

Section: Related Approachessupporting

confidence: 89%

SeMoDe – Simulation and Benchmarking Pipeline for Function as a Service

Manner¹

2021

Bamberger Beiträge Zur Wirtschaftsinformatik Und Angewandten Informatik

View full text Add to dashboard Cite

Cloud computing started with the promise of delivering computing resources elastically at scale, pay per use and on demand self-service to name a few capabilities. In early 2016, Amazon Web Services (AWS) launched a new product called AWS Lambda which started the so called serverless hype and established a new cloud delivery model, namely Function as a Service (FaaS). FaaS offerings keep the promise of delivering computing resources on demand. They dynamically scale up and down function instances and introduce the most fine-grained billing model across all as-a-service offerings by accounting on a milliseconds basis. Despite this flexibility and the possibility to concentrate on the business functionality, a FaaS user loses operational control. Only a few configuration options remain to tune the functions. The first pay-as-you-go billing model raises new questions for performance-cost trade-offs. In order to choose a suitable configuration dependent on the use case and get a solid understanding of performance impact of FaaS platforms, SeMoDe implements a benchmarking and simulation pipeline. It calibrates a physical developer machine, simulates the function in different settings which are comparable to those of cloud offerings and enables a decision guidance to choose an appropriate configuration when deploying it. Based on a Structured Literature Review (SLR) to show the benchmarking and simulation efforts, I suggest a checklist for conducting fair, repeatable and meaningful benchmarks with a focus on documenting the experiments.

show abstract

Section: Related Approachessupporting

confidence: 89%

SeMoDe – Simulation and Benchmarking Pipeline for Function as a Service

Manner¹

2021

Bamberger Beiträge Zur Wirtschaftsinformatik Und Angewandten Informatik

View full text Add to dashboard Cite

show abstract

“…FaaS dynamically deploys multiple, less correlated computations, but the cost-performance evaluations have been based on functions like floating-point operations, numerical calculations, and file conversions. 6,7 To software engineers, these functions are value neutral. Our work not only automates the FaaSification of a concrete software engineering task, but it also illuminates the value-driven evaluation of serverless computing in terms of bug detection and computation cost.…”

Section: Resultsmentioning

confidence: 99%

“…As FaaS makes the charge on a per-invocation basis, 7 it is crucial to decide which functions to execute in parallel in the container and which ones to deploy in a later round of service computation. We define round to be a unit that runs a subset of the metamorphic testing functions in the FaaS container, and the remaining functions have either completed their executions or are waiting in the queue.…”

Section: Function Invocationmentioning

confidence: 99%

Scientific Software Testing Goes Serverless: Creating and Invoking Metamorphic Functions

2021

View full text Add to dashboard Cite

Our function-as-a-service (FaaS) framework transformed end users' questions into automated tests for scientific software. Our case study illustrates the FaaSification of scientific software testing and the importance of value-driven evaluations by focusing on real-world defect detection.MANY SCIENTIFIC RESEARCHERS rely on software to perform their research. In climate research, for instance, scientists and policy makers rely on code simulations to inform their recommendations and decisions. 1 Because scientific software needs to produce trustworthy results, rigorous software engineering practices should be adopted to assure quality.Although testing is a mainstream approach toward software quality, scientific software developers face many challenges, 2 one of which is that only a small group of scientific domain experts (SDEs) who are software developers themselves could write the tests in relation to the scientific theories and models, and yet their primary goal is to further science. 3 Fortunately, science is a community endeavor, and the user community is the cornerstone underlying the sustainable development of any scientific software. However, without the proper tools, serving the user community poses additional burdens on the SDE developers.To alleviate the burdens, we present a function-as-a-service (FaaS) framework aiming to support the SDEs in efficiently processing software users' questions and further transforming the questions into automatic tests. Because users raise their concerns on an asneeded basis, building a monolithic service (for example, via the traditional web clientserver architecture) with availability and capacity would not be cost-effective. FaaS breaks the monolith into functions that can be separately instantiated, executed, and destroyed, and hence, reduces the operational cost. The functions in our work are extracted from the software forum's posts and transformed into system-level test cases. For example, the two functions, increase_bio_retention_cells() and decrease_ runoff(), are generated from the post expressing, "the simulated surface water runoff is expected to decrease when bioretention cells are added to an area of study."

show abstract

“…We leveraged the official boto3 library to interface with the AWS API making the request directly to the system (instead of using API gateway) to better analyze the behaviour of the AWS Lambda itself. The code performing the experiments, as well as the analysis, is publicly available on the project's Github repository 3 .…”

Section: Methodsmentioning

confidence: 99%

“…Since there is very little official documentation made publicly available about the operation and management of serverless offerings by cloud vendors, this information needs to be gathered through experimentation and reverse-engineering these offerings. Thus, we decided to use the results of previous work done in this area [3,5,8,12], as well as our own set of experiments to determine several important details about the operation and management of the public serverless computing platforms. In this section, we will go through our findings as a foundation for the proposed analytical model.…”

Section: System Descriptionmentioning

confidence: 99%