X-RDMA: Effective RDMA Middleware in Large-scale Production Environments

Ma, Teng; Ma, Tao; Song, Zhuo; Jingxuan, Li; Huaixin, Chang; Kang, Changeon; Jiang, Hai; Wu, Yongwei

doi:10.1109/cluster.2019.8891004

Cited by 16 publications

(11 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main difference among these works is that they propose to build this new interface layer at different locations: in kernel space (LITE [120]) or in userspace (X-RDMA [121], nethuns [112], INSANE [113], and Demikernel [111]).…”

Section: High-level Apimentioning

confidence: 99%

Empowering Cloud Computing With Network Acceleration: A Survey

Rosa,

Foschini,

Corradi

2024

IEEE Commun. Surv. Tutorials

View full text Add to dashboard Cite

Modern interactive and data-intensive applications must operate under demanding time constraints, prompting a shift toward the adoption of specialized software and hardware network acceleration technologies. This specialization, however, poses significant scalability, flexibility, security, and economic sustainability challenges for application developers. Cloud computing holds the potential to overcome these obstacles by offering the cost-effective option to access specialized acceleration technologies through standard cloud interfaces. Nevertheless, that integration is still challenging for cloud providers. In the cloud, physical resources are hidden behind a virtualization layer, whereas acceleration technologies make applications directly interact with the hardware. To bridge this gap, recent literature explores the possibility of empowering cloud platforms with accelerated networking as a commodity, thus offering the innovative option of Network Acceleration as a Service. This survey reviews these recent research efforts by adopting popular technologies like XDP, DPDK, and RDMA as a reference. To organize the surveyed research in a comprehensive framework, we identify four key aspects that pose critical problems to the integration of acceleration options in cloud computing -access interfaces, virtualization techniques, serviceability, and security -and systematically discuss the associated challenges. Then, we present the issues to be further addressed and outline the most promising research directions for the full integration of network acceleration within next-generation cloud computing platforms.

show abstract

Section: High-level Apimentioning

confidence: 99%

Empowering Cloud Computing With Network Acceleration: A Survey

Rosa,

Foschini,

Corradi

2024

IEEE Commun. Surv. Tutorials

View full text Add to dashboard Cite

show abstract

“…Hybrid (event + busy) [30] switches between event mode and busy polling with static length of timer. It has two drawbacks with two types of WC load pattern.…”

Section: Limitation In Wc Handling Approachesmentioning

confidence: 99%

“…We set one channel(QP) per remote node and each channel has a CQ except SCQ(Shared CQ) case. Note that Hybrid case in [30] does not have open source code nor algorithm in the paper, it is not included in the comparison.…”

Section: Adaptive Pollingmentioning

confidence: 99%

“…Since Accelio does not provide node level abstraction, we build the same Remote File System with our FUSE-based implementation by replacing network stack with Accelio. Remote regions [44], X-RDMA [30] and LITE [29] are also RDMA abstraction for remote memory access but RemoteRegion and X-RDMA don't have open source code and LITE requires modification of OS. So they don't fit for our purpose.…”

Section: Comparison Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

RDMAbox : Optimizing RDMA for Memory Intensive Workloads

Bae,

Liu,

et al. 2021

Preprint

View full text Add to dashboard Cite

We present RDMAbox, a set of low level RDMA optimizations that provide better performance than previous approaches. The optimizations are packaged in easy-to-use kernel and userspace libraries and presented through simple node level abstractions. We demonstrate the flexibility and effectiveness of RDMAbox by implementing a kernel remote paging system and a userspace file system using RDMAbox. RDMAbox employs two optimization techniques. First, we suggest Load-aware Batching to further reduce the total number of I/O operations to the RDMA NIC beyond existing doorbell batching. The I/O merge queue at the same time functions as a traffic regulator to enforce admission control and avoid overloading the NIC. Second, we propose Adaptive Polling to achieve higher efficiency of polling Work Completion than existing busy polling while maintaining the low CPU overhead of event trigger. Our implementation of a remote paging system with RDMAbox outperforms existing representative solutions with up to 6.48× throughput improvement and up to 83% decrease in average tail latency in bigdata workloads, and up to 83% reduction in completion time in machine learning workloads. Our implementation of a userspace file system based on RDMAbox achieves up to 6× higher throughput over existing representative solutions.

show abstract

“…First, LITE still pays the initialization cost when there is a cache miss. Second, caching all RDMA connections to all nodes is resource inefficient (e.g., taking several GBs of memory), especially when a production RDMA-capable cluster has reached a scale of more than 10,000 nodes [30]. Finally, sharing RDMA connection complicates the preservation of its low-level interfaces, which is important to apply the broadly-explored RDMA-aware optimizations [59,49,13,50,22,23].…”

Section: Introductionmentioning

confidence: 99%

KRCORE: a microsecond-scale RDMA control plane for elastic computing

Wei¹,

Lu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents KRCORE, an RDMA library with a microsecond-scale control plane on commodity RDMAcapable network interface cards. KRCORE can establish a network connection to any node in the cluster within 10µs (hundreds or thousands of times faster than existing approaches), while only maintaining a (small) fixed-size connection metadata at each node, regardless of the cluster scale. The key ideas include virtualizing a pre-initialized kernel-space RDMA connection instead of creating one from scratch, and retrofitting advanced RDMA dynamic connected transport with static transport for both low connection overhead and high resource utilization. KRCORE can shorten the worker bootstrap time of existing disaggregated key-value store (namely RACE Hashing) by 83% under load spikes. In serverless computing (namely Fn), another popular elastic and ephemeral computing system, KRCORE can reduce the latency for transferring data through RDMA by 99%.

show abstract

X-RDMA: Effective RDMA Middleware in Large-scale Production Environments

Cited by 16 publications

References 38 publications

Empowering Cloud Computing With Network Acceleration: A Survey

Empowering Cloud Computing With Network Acceleration: A Survey

RDMAbox : Optimizing RDMA for Memory Intensive Workloads

KRCORE: a microsecond-scale RDMA control plane for elastic computing

Contact Info

Product

Resources

About