Stateful Serverless Computing with Crucial

Abstract

Serverless computing greatly simplifies the use of cloud resources. In particular, Function-as-a-Service (FaaS) platforms enable programmers to develop applications as individual functions that can run and scale independently. Unfortunately, applications that require fine-grained support for mutable state and synchronization, such as machine learning (ML) and scientific computing, are notoriously hard to build with this new paradigm. In this work, we aim at bridging this gap. We present Crucial , a system to program highly-parallel stateful serverless applications. Crucial retains the simplicity of serverless computing. It is built upon the key insight that FaaS resembles to concurrent programming at the scale of a datacenter. Accordingly, a distributed shared memory layer is the natural answer to the needs for fine-grained state management and synchronization. Crucial allows to port effortlessly a multi-threaded code base to serverless, where it can benefit from the scalability and pay-per-use model of FaaS platforms. We validate Crucial with the help of micro-benchmarks and by considering various stateful applications. Beyond classical parallel tasks (e.g., a Monte Carlo simulation), these applications include representative ML algorithms such as k -means and logistic regression. Our evaluation shows that Crucial obtains superior or comparable performance to Apache Spark at similar cost (18%–40% faster). We also use Crucial to port (part of) a state-of-the-art multi-threaded ML library to serverless. The ported application is up to 30% faster than with a dedicated high-end server. Finally, we attest that Crucial can rival in performance with a single-machine, multi-threaded implementation of a complex coordination problem. Overall, Crucial delivers all these benefits with less than 6% of changes in the code bases of the evaluated applications.

Enhancing Resource Utilization Efficiency in Serverless Education: A Stateful Approach with Rofuse

“…It uses local code to manage data storage and network paths, ensuring performance; it interacts with data using a V8-specific intermediate representation to avoid expensive cross-protection domain calls and data copying [10]. Daniel et al proposed a stateful JVM-based serverless programming framework called Crucial, which introduces a distributed shared memory layer in serverless and uses RDMA for inter-node communication to minimize the performance overhead of state exchange between function instances [11]. Sreekanti et al proposed a stateful serverless platform called Cloudburst, which uses an auto-scaling key-value store for efficient statesharing [12].…”

“…The above research solutions mostly explore the implementation of stateful serverless based on specific language runtimes. For example, Crucial [11] is built on JVM; Shredder [10] is built on the V8 engine; Faasm [13] is implemented based on WebAssembly. Moreover, most state-sharing schemes require the applications deployed on them to modify their code using the platform's proprietary APIs, which brings significant invasiveness to user-submitted tasks.…”

Serverless on Machine Learning: A Systematic Mapping Study

“…Compared with cluster computing, a serverless base model enables a rapid adjustment on-demand of the number of workers overtime [62]. Moreover, in multithread computation, a single-machine solution quickly degrades when the number of threads exceeds the number of available cores, while in a serverless base solution, the scale-up is faster regarding the execution time [67].…”

FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds

“…However, SWEEP does not support FCs with branches and is limited to a single region of AWS. Finally, CRUCIAL [22] offers a novel programming model to orchestrate functions in Java by introducing the concept of cloud thread, which corresponds to a single cloud function. CRUCIAL relies on AWS Lambda and uses its retry techniques for fault tolerance.…”

“…Because serverless functions are in general stateless, they do not keep the state of executed code in case of a failure. This causes that the above-mentioned techniques are not applicable if functions increase or decrease value of some data item stored in a database [22]. Therefore, the presented reactive measures are valid for idempotent functions, i.e., functions whose result is not affected if they are executed multiple times.…”