Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

Abstract

The ongoing shift of cloud services from monolithic designs to microservices creates high demand for efficient and high performance datacenter networking stacks, optimized for fine-grained workloads. Commodity networking systems based on software stacks and peripheral NICs introduce high overheads when it comes to delivering small messages.We present Dagger, a hardware acceleration fabric for cloud RPCs based on FPGAs, where the accelerator is closely-coupled with the host processor over a configurable memory interconnect. The three key design principle of Dagger are: (1) offloading the entire RPC stack to an FPGA-based NIC, (2) leveraging memory interconnects instead of PCIe buses as the interface with the host CPU, and (3) making the acceleration fabric reconfigurable, so it can accommodate the diverse needs of microservices. We show that the combination of these principles significantly improves the efficiency and performance of cloud RPC systems while preserving their generality. Dagger achieves 1.3 − 3.8× higher per-core RPC throughput compared to both highly-optimized software stacks, and systems using specialized RDMA adapters. It also scales up to 84 Mrps with 8 threads on 4 CPU cores, while maintaining state-ofthe-art s-scale tail latency. We also demonstrate that large thirdparty applications, like memcached and MICA KVS, can be easily ported on Dagger with minimal changes to their codebase, bringing their median and tail KVS access latency down to 2.8 − 3.5 us and 5.4 − 7.8 us, respectively. Finally, we show that Dagger is beneficial for multi-tier end-to-end microservices with different threading models by evaluating it using an 8-tier application implementing a flight check-in service.

HiveMind

“…2 showed that congested networks can have a dramatic impact on performance and power efficiency; similar observations have been made for cloud-based services [34,42,45,64,65,79,80,97,118]. The remote memory access framework above reduces the overhead of function communication, however, accelerating traditional RPCbased networking is still required, since the edge devices use RPCs to transfer data to/from the cloud [50,89,92,100,105,118]. For network acceleration, we piggyback on the same hardware platform used above.…”

“…The FPGA-based implementation significantly improves performance by bypassing the host's network stack, and the tight integration between the host and FPGA avoids the overheads of the PCIe interfaces [41,65,79,83,89,99,100]. HiveMind directly leverages the processor's cache coherence protocol to handle dirty data tracking and demand paging with no software involvement, thus reducing the remote memory access overhead.…”

A Hardware-Software Stack for Serverless Edge Swarms

“…2 showed that congested networks can have a dramatic impact on performance and power efficiency; similar observations have been made for cloud-based services [33,41,44,70,71,85,86,103,124]. The remote memory access framework above reduces the overhead of function communication, however, accelerating traditional RPC-based networking is still required, since the edge devices use RPCs to transfer data to/from the cloud [49,95,98,106,111,124]. For network acceleration, we piggyback on the same hardware platform used above.…”

“…The FPGA-based implementation significantly improves performance by bypassing the host's network stack, and the tight integration between the host and FPGA avoids the overheads of the PCIe interfaces [40,71,85,89,95,105,106]. HiveMind directly leverages the processor's cache coherence protocol to handle dirty data tracking and demand paging with no software involvement, thus reducing the remote memory access overhead.…”

Characterizing and synthesizing the workflow structure of microservices in ByteDance Cloud

“…These microservice components are composed via standardized Remote Procedure Calls (RPC) interfaces, such as Google's gRPC 3 or Facebook/Apach's Thrift 4 . The dependencies among components and their communication cost, once insignificant for monolithic applications, can now dominate the microservice regime 5,6 …”