Revisiting network support for RDMA

Mittal, Radhika; Shpiner, Alexander; Panda, Aurojit; Zahavi, Eitan; Krishnamurthy, Arvind; Ratnasamy, Sylvia; Shenker, Scott

doi:10.1145/3230543.3230557

Cited by 154 publications

(48 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By contrast, because it operates only over a single hop, BFC's control loop is shorter and simpler to configure, and can therefore support higher utilization for a given target tail latency. Selective retransmission: We can remove the need for all-or-nothing PFC pause frames for RDMA by changing the standard to support selective retransmission in hardware, as suggested by IRN [27]. Handling arbitrary out-of-order packets is non-trivial in hardware, however, and flows can still experience deep packet buffers and packet loss, especially during incast events.…”

Section: Related Workmentioning

confidence: 99%

Backpressure Flow Control

Goyal¹,

Shah

Sharma

et al. 2019

Proceedings of the 2019 Workshop on Buffer Sizing

View full text Add to dashboard Cite

Effective congestion control in a multi-tenant data center is becoming increasingly challenging with rapidly increasing workload demand, ever faster links, small average transfer sizes, extremely bursty traffic, limited switch buffer capacity, and one-way protocols such as RDMA. Existing deployed algorithms, such as DCQCN, are still far from optimal in many plausible scenarios, particularly for tail latency. Many operators compensate by running their networks at low average utilization, dramatically increasing costs.In this paper, we argue that we have reached the practical limits of end-to-end congestion control. Instead, we propose a new clean slate design based on hop-by-hop per-flow flow control. We show that our approach achieves near optimal tail latency behavior even under challenging conditions such as high average link utilization and in-cast cross traffic. By contrast with prior hop-by-hop schemes, our main innovation is to show that per-flow flow control can be achieved with limited metadata and packet buffering. Further, we show that our approach generalizes well to cross-data center communication.

show abstract

Section: Related Workmentioning

confidence: 99%

Backpressure Flow Control

Goyal¹,

Shah

Sharma

et al. 2019

Proceedings of the 2019 Workshop on Buffer Sizing

View full text Add to dashboard Cite

show abstract

“…PFC can achieve a lossless network when configured correctly. PFC is a coarse-grained mechanism which can lead to some problems as shown in Reference [21]. For example, as shown in Figure 1, when the congestion occurs in the S1 switch and the queue size of the ingress port exceeds the threshold, a PFC PAUSE frame will be sent to the upstream switch (L1) due to the cascading effect of PAUSE frame and switch L1 stops sending data to switch S1.…”

Section: Priority Flow Controlmentioning

confidence: 99%

“…Some other congestion control protocols are also proposed, such as DCQCN [12], TIMELY [25] and IRN [21]. DCQCN is a rate-based, end-to-end congestion control protocol, that builds upon QCN and DCTCP.…”

Section: Other Relevant Protocolmentioning

confidence: 99%

See 1 more Smart Citation

P4QCN: Congestion Control Using P4-Capable Device in Data Center Networks

2019

View full text Add to dashboard Cite

Modern data centers aim to offer very high throughput and ultra-low latency to meet the demands of applications such as online intensive services. Traditional TCP/IP stacks cannot meet these requirements due to their high CPU overhead and high-latency. Remote Direct Memory Access (RDMA) is an approach that can be designed to meet this demand. The mainstream transport protocol of RDMA over Ethernet is RoCE (RDMA over Converged Ethernet), which relies on Priority Flow Control (PFC) within the network to enable a lossless network. However, PFC is a coarse-grained protocol which can lead to problems such as congestion spreading, head-of-the-line blocking. A congestion control protocol that can alleviate these problems of PFC is needed. We propose a protocol, called P4QCN for this purpose. P4QCN is a congestion control scheme for RoCE and it is an improved Quantized Congestion Notification (QCN) design based on P4, which is a flow-level, rate-based congestion control mechanism. P4QCN extends the QCN protocol to make it compatible with IP-routed networks based on a framework of P4 and adopts a two-point algorithm architecture which is more effective than the three-point architecture used in QCN and Data Center QCN(DCQCN). Experiments show that our proposed P4QCN algorithm achieves the expected performance in terms of latency and throughput.

show abstract

“…Typically, polling is relying on the last packet of a written message to detect request completion. However, the correctness of this polling-based approach depends on the delivery of the message in order to avoid anticipated last packet delivery or memory overwritten by an older message [18].…”

Section: Communication Paradigmsmentioning

confidence: 99%

Communicating Efficiently on Cluster-Based Remote Direct Memory Access (RDMA) over InfiniBand Protocol

2018

View full text Add to dashboard Cite

Distributed systems are commonly built under the assumption that the network is the primary bottleneck, however this assumption no longer holds by emerging high-performance RDMA enabled protocols in datacenters. Designing distributed applications over such protocols requires a fundamental rethinking in communication components in comparison with traditional protocols (i.e., TCP/IP). In this paper, communication paradigms in existing systems and new possible paradigms have been investigated. Advantages and drawbacks of each paradigm have been comprehensively analyzed and experimentally evaluated. The experimental results show that writing the requests to server and reading the response presents up to 10 times better performance comparing to other communication paradigms. To further expand the investigation, the proposed communication paradigm has been substituted in a real-world distributed application, and the performance has been enhanced up to seven times.

show abstract

Revisiting network support for RDMA

Cited by 154 publications

References 25 publications

Backpressure Flow Control

Backpressure Flow Control

P4QCN: Congestion Control Using P4-Capable Device in Data Center Networks

Communicating Efficiently on Cluster-Based Remote Direct Memory Access (RDMA) over InfiniBand Protocol

Contact Info

Product

Resources

About