Communicating Efficiently on Cluster-Based Remote Direct Memory Access (RDMA) over InfiniBand Protocol

Abstract

Distributed systems are commonly built under the assumption that the network is the primary bottleneck, however this assumption no longer holds by emerging high-performance RDMA enabled protocols in datacenters. Designing distributed applications over such protocols requires a fundamental rethinking in communication components in comparison with traditional protocols (i.e., TCP/IP). In this paper, communication paradigms in existing systems and new possible paradigms have been investigated. Advantages and drawbacks of each paradigm have been comprehensively analyzed and experimentally evaluated. The experimental results show that writing the requests to server and reading the response presents up to 10 times better performance comparing to other communication paradigms. To further expand the investigation, the proposed communication paradigm has been substituted in a real-world distributed application, and the performance has been enhanced up to seven times.

In-Network Monitoring Strategies for HPC Cloud

“…Communications over traditional LAN technologies and the TCP/IP protocol stack are unfit for HPC networks because of inferior performance characteristics and significant CPU and memory usage. Consequently, a new category of network fabrics emerged, using a technology known as Remote Direct Memory Access (RDMA) [6]. RDMA is a high-performance networking technology that allows direct memory access between computers in a network, bypassing traditional network stack layers and reducing latency.…”