IceBreaker: warming serverless functions better with heterogeneity

Abstract

Serverless computing, an emerging computing model, relies on łwarming upž functions prior to its anticipated execution for faster and cost-effective service to users. Unfortunately, warming up functions can be inaccurate and incur prohibitively expensive cost during the warmup period (i.e., keep-alive cost). In this paper, we introduce IceBreaker, a novel technique that reduces the service time and the łkeep-alivež cost by composing a system with heterogeneous nodes (costly and cheaper). IceBreaker does so by dynamically determining the cost-effective node type to warm up a function based on the function's time-varying probability of the next invocation. By employing heterogeneity, IceBreaker allows for more number of nodes under the same cost budget and hence, keeps more number of functions warm and reduces the wait time during high load. Our real-system evaluation confirms that IceBreaker reduces the overall keep-alive cost by 45% and execution time by 27% using representative serverless applications and industry-grade workload trace. IceBreaker is the first technique to employ and leverage the idea of mixing expensive and cheaper nodes to improve both service time and keep-alive cost for serverless functions ś opening up a new research avenue of serverless computing on heterogeneous servers for researchers and practitioners. CCS CONCEPTS• Computer systems organization → Cloud computing; ntier architectures; • General and reference → Performance.

Latency And Resource Consumption Analysis For Serverless Edge Analytics

“…There exist also some plugins, such as the Serverless WarmUp plugin [24] for AWS Lambda, which creates a scheduled lambda that invokes all the selected service's lambdas in a configured time interval (5 minutes, by default), forcing the lambda function instances to stay warm. Some other improvements to the basic keep-alive mechanism have been proposed, for example in [25] the keep-alive interval is adapted to each particular workload, according to its actual invocation frequency and pattern, [26] uses cachingbased techniques to implement a greedy-dual keep-alive policy based on the memory footprint, access frequency, initialization cost, and execution latency of different functions, and [27] proposes the IceBreaker technique, which reduces the service time and the keep-alive cost by composing a system with heterogeneous nodes (costly and cheaper), by dynamically determining the cost-effective node type to warm up a function based on the function's time-varying probability of the next invocation. Another technique for reducing the cold start problem is instance pre-warming, which consists of starting in advance a given number of function instances that stay always alive during the serverless application lifecycle, and can run different invocations of the same function.…”

AQUATOPE: QoS-and-Uncertainty-Aware Resource Management for Multi-stage Serverless Workflows

“…Cold starts: Cold starts are one of the most studied overheads associated with serverless [31,54,56,63,65]. A cold start invocation occurs when a serverless application is triggered, but its function instances are not yet loaded in memory.…”

“…Shahrad et al [57] proposed a histogram-based policy to adjust a container's keep-alive time. Similarly, IceBreaker [54] uses a Fourier-transformation-based model to predict future invocation patterns, and pre-warms function containers accordingly. These techniques can mitigate cold starts, but are often not robust to fluctuating workloads, and are designed for single-stage serverless applications, which are not prone to cascading cold starts across dependent functions.…”

“…(3) FaaSCache's container eviction and dynamic auto-scaling policy [31]; (4) histogram-based container keep-alive policy in [57], which uses historical function inter-arrival time to dynamically adjust the keep-alive time; (5) Icebreaker [54], which uses Fourier Transformation to predict and pre-warm function containers based on historical invocation patterns.…”

“…Therefore, FaaSCache falls back to a conservative dynamic auto-scaling policy. The histogram-based method in [57] and IceBreaker [54] use the function invocation inter-arrival time distribution to predict future invocations, and dynamically pre-warm and keep-alive containers. They outperform autoscaling and further eliminate 13%-17% of cold starts.…”