Evaluation of Production Serverless Computing Environments

Lee, Hyungro; Satyam, Kumar; Fox, Geoffrey

doi:10.1109/cloud.2018.00062

Cited by 146 publications

(96 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Numerous works evaluate commercial and open source FaaS solutions [11,15,17] focusing on analysis of supported features and comparison based on the identified feature sets. Leitner et al [16] presents a mixed-method empirical study focusing on FaaS development in industrial practice, which combines a gray literature analysis with interviews and online questionnaire.…”

Section: Related Workmentioning

confidence: 99%

A Systematic Mapping Study on Engineering Function-as-a-Service Platforms and Tools

Yussupov

Breitenbücher

Leymann

et al. 2019

Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

Function-as-a-Service (FaaS) is a novel cloud service model allowing to develop fine-grained, provider-managed cloud applications. In this work, we investigate which challenges motivate researchers to introduce or enhance FaaS platforms and tools. We use a systematic mapping study method to collect and analyze the relevant scientific literature, which helps us answering the three clearly-defined research questions. We design our study using well-established guidelines and systematically apply it to 62 selected publications. The collected and synthesized data provides useful insights into the main challenges that motivate researchers to work on this topic and can be helpful in identifying research gaps for future research.

show abstract

Section: Related Workmentioning

confidence: 99%

A Systematic Mapping Study on Engineering Function-as-a-Service Platforms and Tools

Yussupov

Breitenbücher

Leymann

et al. 2019

Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

show abstract

“…Back and Andrikopoulos [16] as well as Lee et al [17] did performance testing to compare FaaS offerings of different providers. They did this by gradually incrementing load on a single function while observing resource usage and comparing costs.…”

Section: Serverless Computing Performancementioning

confidence: 99%

“…Some papers [10][11][12] look at possible use-case for Serverless by considering their advertised benefits and costs. Others [16][17][18] investigated FaaS performance in-depth and across multiple cloud providers. We see an opportunity here to investigate a specific and common use-case for FaaS, namely Web APIs.…”

Section: Serverless Computing Performancementioning

confidence: 99%

Serverless Computing: An Investigation of Deployment Environments for Web APIs

Ivan

Radu²,

Dadarlat

2019

Computers

View full text Add to dashboard Cite

Cloud vendors offer a variety of serverless technologies promising high availability and dynamic scaling while reducing operational and maintenance costs. One such technology, serverless computing, or function-as-a-service (FaaS), is advertised as a good candidate for web applications, data-processing, or backend services, where you only pay for usage. Unlike virtual machines (VMs), they come with automatic resource provisioning and allocation, providing elastic and automatic scaling. We present the results from our investigation of a specific serverless candidate, Web Application Programming Interface or Web API, deployed on virtual machines and as function(s)-as-a-service. We contrast these deployments by varying the number of concurrent users for measuring response times and costs. We found no significant response time differences between deployments when VMs are configured for the expected load, and test scenarios are within the FaaS hardware limitations. Higher numbers of concurrent users or unexpected user growths are effortlessly handled by FaaS, whereas additional labor must be invested in VMs for equivalent results. We identified that despite the advantages serverless computing brings, there is no clear choice between serverless or virtual machines for a Web API application because one needs to carefully measure costs and factor-in all components that are included with FaaS.

show abstract

“…From more related works, [7] and [10] set out to explicitly benchmark and compare FaaS solutions in terms of performance and cost. While useful and insightful in their own right, both works use much more coarse-grained tasks for their evaluation, focusing on concurrency and latency, respectively.…”

Section: Background and Related Workmentioning

confidence: 99%

Using a Microbenchmark to Compare Function as a Service Solutions

Back

Andrikopoulos

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The Function as a Service (FaaS) subtype of serverless computing provides the means for abstracting away from servers on which developed software is meant to be executed. It essentially offers an eventdriven and scalable environment in which billing is based on the invocation of functions and not on the provisioning of resources. This makes it very attractive for many classes of applications with bursty workload. However, the terms under which FaaS services are structured and offered to consumers uses mechanisms like GB-seconds (that is, X GigaBytes of memory used for Y seconds of execution) that differ from the usual models for compute resources in cloud computing. Aiming to clarify these terms, in this work we develop a microbenchmark that we use to evaluate the performance and cost model of popular FaaS solutions using well known algorithmic tasks. The results of this process show a field still very much under development, and justify the need for further extensive benchmarking of these services.

show abstract

Evaluation of Production Serverless Computing Environments

Cited by 146 publications

References 4 publications

A Systematic Mapping Study on Engineering Function-as-a-Service Platforms and Tools

A Systematic Mapping Study on Engineering Function-as-a-Service Platforms and Tools

Serverless Computing: An Investigation of Deployment Environments for Web APIs

Using a Microbenchmark to Compare Function as a Service Solutions

Contact Info

Product

Resources

About