Occupancy Sampling for Terabit CEE Switches

Abstract

One consequential feature of Converged Enhanced Ethernet (CEE) is losslessness, achieved through L2 Priority Flow Control (PFC) and Quantized Congestion Notification (QCN).We focus on QCN and its effectiveness in identifying congestive flows in input-buffered CEE switches. QCN assumes an idealized, output-queued switch; however, as future switches scale to higher port counts and link speeds, purely output-queued or sharedmemory architectures lead to excessive memory bandwidth requirements; moreover, PFC typically requires dedicated buffers per input.Our objective is to complement PFC's coarse per-port/priority granularity with QCN's per-flow control. By detecting buffer overload early, QCN can drastically reduce PFC's side effects. We install QCN congestion points (CPs) at input buffers with virtual output queues and demonstrate that arrival-based marking cannot correctly discriminate between culprits and victims.Our main contribution is occupancy sampling (QCN-OS), a novel, QCN-compatible marking scheme. We focus on random occupancy sampling, a practical method not requiring any perflow state. For CPs with arbitrarily scheduled buffers, QCN-OS is shown to correctly identify congestive flows, improving buffer utilization, switch efficiency, and fairness.

Integration and QoS of multicast traffic in a server-rack fabric with 640 100g ports

“…In this experiment, we used the QCN-standard flow marking scheme, which sends the CNMs to the most recently received frames. In our design, we have corrected this shortcoming with an occupancy sampling flow-marking scheme for QCN, which sends the CNMs to the flow with the largest occupancy in the monitored payload buffer pool [5]. Figures 10(c,d) demonstrate that our rectified flow-marking scheme, with congestion points at the inputs, performs in par with standard QCN at switch outputs.…”

“…In a previous publication [5], we found that, for unicast traffic, placing the congestion points at the inputs instead of outputs improves QCN's fairness and reduces the bandwidth overhead of CNMs. Furthermore, for multicast traffic, this alternative will generate at most one CNM per multicast frame, whatever the fanout of the flow may be.…”

“…Every leaf constitutes the networkedge point for five (5) servers, offering 100 Gb/s bandwidth to each on a single Ethernet link. Additionally it offers one surrogate and two PCIe ports, for a total of eight (8) main ports.…”

“…First, we demonstrate that the industry-standard method, Quantized Congestion Notifications (QCN), does not work properly with one-to-many multicast flows. Then we demonstrate that our congestion management architecture, first presented and evaluated for unicast traffic in [5], which (a) places the QCN congestion points at fabric inputs instead of fabric outputs, and (b) modifies the standard-QCN marking scheme, works smoothly with multicast flows.…”

Non‐minimal adaptive routing based on explicit congestion notifications

“…• Congestion detection at switch input buffers: Neeser et al 14 suggested the use of input-buffer occupancy as a congestion indicator, which provides better fairness and improved detection of offending and victim flows. When generating a CNM, they also selected a victim packet randomly from the complete buffer, not necessarily the packet in the head, in order to improve fairness.…”