An Evaluation of Open Source Serverless Computing Frameworks

Abstract

Recent advancements in virtualization and software architecture have led to the new paradigm of serverless computing, which allows developers to deploy applications as stateless functions without worrying about the underlying infrastructure. Accordingly, a serverless platform handles the lifecycle, execution and scaling of the actual functions; these need to run only when invoked or triggered by an event. Thus, the major benefits of serverless computing are low operational concerns and efficient resource management and utilization. Serverless computing is currently offered by several public cloud service providers. However, there are certain limitations on the public cloud platforms, such as vendor lock-in and restrictions on the computation of the functions. Open source serverless frameworks are a promising solution to avoid these limitations and bring the power of serverless computing to on-premise deployments. However, these frameworks have not been evaluated before. Thus, we carry out a comprehensive feature comparison of popular open source serverless computing frameworks. We then evaluate the performance of selected frameworks: Fission, Kubeless and OpenFaaS. Specifically, we characterize the response time and ratio of successfully received responses under different loads and provide insights into the design choices of each framework.

An Evaluation of Open Source Serverless Computing Frameworks Support at the Edge

“…Open-source serverless frameworks have been introduced to avoid the vendor lock-in. Mohanty et al [18] performed a feature comparison of four open source serverless frameworks (Kubless, OpenFaaS, Fission, and OpenWhisk). The authors also evaluated the performance of three frameworks (excluding OpenWhisk) when deployed on Kubernetes cluster.…”

“…Currently, all major cloud service providers offer serverless computing platforms (e.g., AWS Lambda [14], Azure Functions [15], IBM Cloud Functions [16], and Cloud Functions [17]). However, such platforms require functions to be written or deployed in a certain way, which results in vendor lock-in [18]. Several open-source FaaS frameworks have been proposed to allow to run serverless computing on private infrastructure, thereby avoiding any forms of vendor lockin.…”

“…Several open-source FaaS frameworks have been proposed to allow to run serverless computing on private infrastructure, thereby avoiding any forms of vendor lockin. Recent studies such as Mohanty et al [18] and Kritikos et al [19] have evaluated the usefulness and performance of selected open source serverless frameworks, but such studies have not considered the constraints introduced by an edgebased environment.…”

Understanding Open Source Serverless Platforms

“…The works [5,12] investigate the different factors that influence the performance of AWS lambda, namely the impact of the choice of language of the function, memory footprint of the function, etc. Work [7] evaluates the performance of Fission, Kubeless and OpenFaaS serverless frameworks and characterizes the response time and the ratio of successfully completed requests for different loads. However the work fails to characterize the throughput of these platforms and accounts for the mean latency (response time) and successful responses at different load characteristics, which is debatable, without the proper consideration and configuration of the serverless platform specific configuration parameters, resulting in inaccurate results.…”

Function-as-a-Service Benchmarking Framework

“…For this purpose, the authors develop a microbenchmark in order to investigate the observable behavior with respect to the computer/memory relation of different FaaS platforms, and the pricing models currently being in use. Mohanty et al (2018) analyse the status of Open-source serverless computing frameworks Fission (2019), Kubeless (2019) and OpenFaaS (2019). For this purpose, the authors evaluate the performance of the response time and ratio of successfully responses under different loads.…”