Analyzing Tail Latency in Serverless Clouds with STeLLAR

Abstract

Serverless computing has seen rapid adoption because of its instant scalability, flexible billing model, and economies of scale. In serverless, developers structure their applications as a collection of functions invoked by various events like clicks, and cloud providers take responsibility for cloud infrastructure management. As with other cloud services, serverless deployments require responsiveness and performance predictability manifested through low average and tail latencies. While the average end-to-end latency has been extensively studied in prior works, existing papers lack a detailed characterization of the effects of tail latency in real-world serverless scenarios and their root causes.In response, we introduce STeLLAR, an open-source serverless benchmarking framework, which enables an accurate performance characterization of serverless deployments. STeLLAR is provider-agnostic and highly configurable, allowing the analysis of both end-to-end and per-component performance with minimal instrumentation effort. Using STeLLAR, we study three leading serverless clouds and reveal that storage accesses and bursty function invocation traffic are key factors impacting tail latency in modern serverless systems. Finally, we identify important factors that do not contribute to latency variability, such as the choice of language runtime.

Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications

“…Hence, serverless fulfills its promise of built-in scalability under the given load levels for our 10 applications. This result was somewhat unexpected, especially contrasting with prior research [10,38,72]. However, of course, bursty workloads may still negatively impact performance on different platforms or with even more rapid bursts than what we evaluated (e.g., per-microsecond bursts rather than per-second bursts).…”

“…Latency-critical applications should carefully choose external services and trigger types. Measurement studies covering different external services can guide the initial selection process [38,55,72,81]. Our results confirm these findings and identify cloud storage as key contributor to performance variability [72].…”

“…Measurement studies covering different external services can guide the initial selection process [38,55,72,81]. Our results confirm these findings and identify cloud storage as key contributor to performance variability [72]. Beyond that, sb can provide insights about alternative application implementations.…”

“…Tail latency is increasingly important at scale for cloud providers [19] and hence particularly challenging for massive multi-tenant serverless systems. Prior studies [38,55,72,81] conducted micro-benchmarks to measure tail of individual serverless components. By leveraging our trace analysis (Section 4), we can directly identify which time categories contribute to tail latency (99th percentile) for entire applications.…”

“…Real-world applications exhibit diverse invocation patterns [66] but prior work rarely investigated dynamic workloads over time [38] or different invocation patterns [37] and if so, using artificial applications, patterns, and one-time bursts [10,72,81]. It remains unclear how different invocation patterns derived from the Azure Function Traces [66] (Section 3.4) affect the end-to-end latency of serverless applications.…”