Turboflow

Abstract

Fine-grained traffic flow records enable many powerful applications, especially in combination with telemetry systems that supports high coverage, i.e., of every link and at all times. Current solutions, however, make undesirable trade-offs between infrastructure cost and information richness. Switches that generate flow records, e.g., NetFlow switches, are a low cost solution but current designs sacrifice information richness, e.g., by sampling. Information rich alternatives rely heavily on servers, which increases cost to the point that they are impractical for high coverage. In this paper, we present the design, implementation, and evaluation of TurboFlow, a flow record generator for programmable switches that does not compromise on either cost or information richness. TurboFlow produces fine-grained and unsampled flow records with custom features entirely at the switch without relying on any support from external servers. This is a challenge given high traffic rates and the limitations of switch hardware. To overcome, we decompose the flow record generation algorithm and optimize it for the heterogeneous processors in programmable switches. We show that with this design, TurboFlow can support multi-terabit workloads on readily available commodity switches to enable information rich monitoring with high coverage.

Congestion Control for Datacenter Networks: A Control-Theoretic Approach

“…Our flow table is implemented using a simple Register array in P4. It is similar to the flow table used in Turboflow [21], which is proven not to hinder line-rate traffic processing in the data plane. The flow table is updated for each data packet going through the egress pipeline; and (ii) set queue length and other parameters required for rate computation (see § III-F) on CNP and "mirror" it to selected sources based on the flow table.…”

“…Concerns with switch-driven solutions that the turnaround time of new features in switch ASICs (application specific integrated circuits) is high are being overturned by the increasing adoption of programmable switch application specific integrated circuits (ASICs) with P4 support [18] by leading switch manufacturers. Similarly, reservations that switch-driven congestion control can hinder line-rate packet processing are countered by recent work [19], [20], [21] showing that event processing (beyond packet arrival and departure events) using P4 does not sacrifice line-rate packet processing.…”

High Performance Network Metadata Extraction Using P4 for ML-based Intrusion Detection Systems

“…A more complete approach is taken by P4ID [26], which compiles stateless intrusion detection rules directly to a P4-enabled switch, while redirecting a certain amount of packets perow for stateful inspection to an IDS, hence reducing the overall load on this system. The aspect of aggregatingow-based data is shown in [27], which utilizes hash collisions during the recording of statistics as trigger to export the collected metadata. In contrast to INT, we emit the extracted data out-ofband via a dedicated port due to security considerations.…”

Adaptive Telemetry for Software-Defined Mobile Networks

“…Traditional tools to implement passive monitoring for IP networks are sFlow [29], netFlow [10] and IPFIX [11]. Modern approaches overcoming the limitations of traditional tools are proposed by Sonchack et al [31,32], Tilmans et al [33], and Gupta et al [14]. These define various techniques for configuring and performing packet sampling, traffic mirroring and statistics aggregation in the network as well as the protocol for transmitting the samples to the central controller (i.e., the packet format) for further processing.…”