Container lifecycle‐aware scheduling for serverless computing

Abstract

Elastic scaling in response to changes on demand is a main benefit of serverless computing. When bursty workloads arrive, a serverless platform launches many new containers and initializes function environments (known as cold starts), which incurs significant startup latency. To reduce cold starts, platforms usually pause a container after it serves a request, and reuse this container for subsequent requests. However, this reuse strategy cannot efficiently reduce cold starts because the schedulers are agnostic of container lifecycle. For example, it may ignore soon available containers or evict soon needed containers. We propose a container lifecycle-aware scheduling strategy for serverless computing, CAS. The key idea is to control distribution of requests and determine creation or eviction of containers according to different lifecycle phases of containers.We implement a prototype of CAS on OpenWhisk. Our evaluation shows that CAS reduces 81% cold starts and therefore brings a 63% reduction at 95th percentile latency compared with native scheduling strategy in OpenWhisk when there is worker contention between workloads, and does not add significant performance overhead.

Efficiency in the serverless cloud paradigm: A survey on the reusing and approximation aspects

“…Containerization: The most common way to package and isolate the function is through containerization. 52 In this technology, a function is encapsulated within a widely-accepted container standard, termed Open Container Initiative (OCI) format, 53 which is supported in all modern containerization solutions. Any programming language and/or software dependency can be supported, thus, the desired generality of serverless is accomplished.…”

Custom Scheduling in Kubernetes: A Survey on Common Problems and Solution Approaches

“…The authors also propose a two-level resource management mechanism, where the higher level entirely leverages the Kubernetes scheduler, whereas the lower level is speciic to function dispatching. [115] presents the concept of łlifecycle-aware schedulingž, where the main idea is to prevent the eviction of containers that may be needed soon and to favor existing containers that will become available sooner than containers that have to be created anew. While this contribution is not directly based on Kubernetes, it leverages the open source serverless platform OpenWhisk [7], which accepts Kubernetes as one of its deployment options.…”

Profiling and controlling I/O‐related memory contention in COTS heterogeneous platforms

“…In the context of virtualization, Wu et al 45 proposed a contained lifecycle‐aware scheduling algorithm for serverless computing, where each computing request is processed in a container with a specified resource requirement in terms of CPU and memory. Other mechanisms have been proposed by Fan et al 46 in the context of cloud computing.…”