Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach

Abstract

Rapid adoption of the 'serverless' (or Function-as-a-Service, FaaS) paradigm [8], pioneered by Amazon with AWS Lambda and followed by numerous commercial offerings and open source projects, introduces new challenges in designing the cloud infrastructure, balancing between performance and cost. While instant per-request elasticity that FaaS platforms typically offer application developers makes it possible to achieve high performance of bursty workloads without over-provisioning, such elasticity often involves extra latency associated with on-demand provisioning of individual runtime containers that serve the functions. This phenomenon is often called 'cold starts ' [12], as opposed to the situation when a function is served by a pre-provisioned 'warm' container, ready to serve requests with close to zero overhead. Providers are constantly working on techniques aimed at reducing cold starts. A common approach to reduce cold starts is to maintain a pool of 'warm' containers, in anticipation of future requests. In this project, we address the cold start problem in serverless architectures, specifically under the Knative Serving FaaS platform. We implemented a pool of function instances and evaluated the latency compared with the original implementation, which resulted in an 85% reduction of P99 response time for a single instance pool.

Serverless Computing: Behind the Scenes of Major Platforms

“…One of the greatest problems facing serverless computing is the infamous cold start. Cold starts have been the subject of research papers and blogs [3], [14], [4], [1], [15], [16] and continue to be a major source of doubt for those considering a serverless based application. They are the notable delay incurred when invoking a function for the first time.…”

Deep reinforcement learning for application scheduling in resource-constrained, multi-tenant serverless computing environments

“…A container management system with three queues containing cold and warm containers based on their features is introduced in [20]. [21], [17], [24] and [18] propose maintaining a pool of function instances to face request demands. A heuristic solution is given in [27] to adjust the replica number without compromising on user budget.…”

Performance Modeling of Serverless Computing Platforms

“…Thus, too many cold starts could impact the application's responsiveness and user experience (Wang et al, 2018). This is the reason a lot of research in the field of serverless computing has focused on mitigating cold starts (Lin and Glikson, 2019;Bermbach et al, 2020;Manner et al, 2018).…”