RFP

Abstract

Remote Direct Memory Access (RDMA) has been widely deployed in modern data centers. However, existing usages of RDMA lead to a dilemma between performance and redesign cost. They either directly replace socket-based send/receive primitives with the corresponding RDMA counterpart (server-reply), which only achieves moderate performance improvement; or push performance further by using one-sided RDMA operations to totally bypass the server (server-bypass), at the cost of redesigning the software.In this paper, we introduce two interesting observations about RDMA. First, RDMA has asymmetric performance characteristics, which can be used to improve server-reply's performance. Second, the performance of server-bypass is not as good as expected in many cases, because more rounds of RDMA may be needed if the server is totally bypassed. We therefore introduce a new RDMA paradigm called Remote Fetching Paradigm (RFP). Although RFP requires users to set several parameters to achieve the best performance, it supports the legacy RPC interfaces and hence avoids the need of redesigning application-specific data structures. Moreover, with proper parameters, it can achieve even higher IOPS than that of the previous paradigms.We have designed and implemented an in-memory keyvalue store based on RFP to evaluate its effectiveness. Experimental results show that RFP improves performance by 1.6⇥⇠4⇥ compared with both server-reply and serverbypass paradigms. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and /or a fee. Request permissions from permissions@acm.org. Figure 1. Improvements of RFP over server-reply and server-bypass. A higher bar means better performance.

Zerializer

“…Towards RPC Offload. RPC performance optimization has been widely studied [5] and has recently seen increased interest due to the trend towards microservices [7,15,33]. Zerializer also allows for offloading RPC.…”

RGraph: Asynchronous graph processing based on asymmetry of remote direct memory access

“…As an emerging network technology, RDMA has become more and more popular in data centers 14,16,18 . RDMA network can provide 200 Gbps throughput and 1 s latency, 14 which brings opportunities for distributed applications.…”

“…However, Su et al 18 observe that one‐sided RDMA operations have asymmetric performance characteristics, that is, the overhead of sending an out‐bound RDMA is much larger than that of receiving an in‐bound RDMA. The reason is that the work of the receiving side is all managed by the hardware of RDMA network interface card (RNIC), while that of the sending side needs both software and hardware to be involved.…”

“…The reason is that the work of the receiving side is all managed by the hardware of RDMA network interface card (RNIC), while that of the sending side needs both software and hardware to be involved. Su et al 18 show through experiments that the IOPS of in‐bound RDMA is five times higher than out‐bound RDMA. The communication between the master vertex and mirror vertices follows a one‐to‐many pattern.…”

“…The large number of small data transfers brings challenges for applying RDMA to asynchronous graph processing systems. Su et al 18 find that the overhead of sending a one‐sided RDMA (i.e., out‐bound RDMA) is much larger than that of receiving a one‐sided RDMA (i.e., in‐bound RDMA). Existing graph partitioning methods 5 usually partition a vertex into multiple replicas and select one of them as the master and others as mirrors.…”