Orchestrating In-Band Data Plane Telemetry With Machine Learning

Hohemberger, Rumenigue; Castro, Ariel Góes De; Vogt, Francisco; Mansilha, Rodrigo B.; Lorenzon, Arthur F.; Rossi, Fábio Diniz; Luizelli, Marcelo Caggiani

doi:10.1109/lcomm.2019.2946562

Cited by 32 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Every individual packet traversing the network carries pertinent information directly to the monitoring system at the line rate. This level of granularity aligns with the perspective presented in [15], wherein it is recognized that a substantial volume of data can prove immensely valuable for Deep Reinforcement Learning (DRL) algorithms, which have a voracious appetite for information.…”

Section: In-band Network Telemetrysupporting

confidence: 69%

Using Machine Learning and In-band Network Telemetry for Service Metrics Estimation

Almeida

Pasquini

Verdi

2021

2021 IEEE 10th International Conference on Cloud Networking (CloudNet)

View full text Add to dashboard Cite

Active Queue Management (AQM) is a mechanism employed to alleviate transient congestion in network device buffers, such as routers and switches. Traditional AQM algorithms use fixed thresholds, like target delay or queue occupancy, to compute random packet drop probabilities. A very small target delay can increase packet losses and reduce link utilization, while a large target delay may increase queueing delays while lowering drop probability. Due to dynamic network traffic characteristics, where traffic fluctuations can lead to significant queue variations, maintaining a fixed threshold AQM may not suit all applications. Consequently, we explore the question: What is the ideal threshold (target delay) for AQMs? In this work, we introduce DESiRED (Dynamic, Enhanced, and Smart iRED), a P4-based AQM that leverages precise network feedback from In-band Network Telemetry (INT) to feed a Deep Reinforcement Learning (DRL) model. This model dynamically adjusts the target delay based on rewards that maximize application Quality of Service (QoS). We evaluate DESiRED in a realistic P4based test environment running an MPEG-DASH service. Our findings demonstrate up to a 90x reduction in video stall and a 42x increase in high-resolution video playback quality when the target delay is adjusted dynamically by DE-SiRED.

show abstract

Section: In-band Network Telemetrysupporting

confidence: 69%

Using Machine Learning and In-band Network Telemetry for Service Metrics Estimation

Almeida

Pasquini

Verdi

2021

2021 IEEE 10th International Conference on Cloud Networking (CloudNet)

View full text Add to dashboard Cite

show abstract

“…Moreover, different users may choose the same devices to transmit their data or some devices may never handle any user packets, which leads to duplication of telemetry or coverage incompleteness of the entire network. It is thus necessary to continue the investigation of the orchestration problem of INT and make an attempt to improve the operation [27].…”

Section: Related Workmentioning

confidence: 99%

Fast and On-demand Network-wide Telemetry via Cluster Based Path Planning

Du Chen,

Deyun Gao

et al. 2023

Journal of Internet Technology

View full text Add to dashboard Cite

<p>In-band Network Telemetry (INT) has profoundly promoted the network visibility. However, the existing solutions either face Maximum Transmission Unit (MTU) limitation or latency issue due to continuous insertion of INT metadata along long forwarding paths. In addition, conducting telemetry using user packets is prone to result in duplicate telemetry and fall short to achieve network-wide telemetry. In this work, we propose NetworkSight to comprehensively resolve these problems through cluster based path planning and source routing based on-demand forwarding. Specifically, a cluster based path planning algorithm is designed to generate several balanced short forwarding paths that cover the entire network. As a consequence, the MTU limitation is overcome and the telemetry latency can be reduced. Instead of operating on user packets, NetworkSight sends crafted probes built on source routing to travel synchronously over the generated forwarding paths. With these improvements, network-wide telemetry is achieved. Besides, we define a suite of telemetry primitives and develop corresponding APIs for users to flexibly express their telemetry requirements. Extensive experimental results show that NetworkSight can generate more balanced forwarding paths. The results show that it outperforms state-of-the-art mechanisms, where it reduces the telemetry latency by tens of milliseconds.</p> <p> </p>

show abstract

“…The goal of INT-path [6] and P2INT [9] is to cover all the networks' links to collect the flow statistics in a P4-based network. The work in [10] concentrates on coordinating of probe packets to get insights from the network flows. However, these approaches focus on coordinating probe packets to collect the flow statistics without considering the overhead of steering probe packets to the collector.…”

Section: Introductionmentioning

confidence: 99%

P4Flow: Monitoring Traffic Flows With Programmable Networks

Mostafaei

Afridi

2021

IEEE Commun. Lett.

View full text Add to dashboard Cite

Cloud providers perform flow monitoring to get insights from the network traffic flows, often using coarse-grained packet counters or packet probing. These approaches give partial information from ongoing flows or introduce significant overhead if the probe packets cross multiple hops with diverse delay and bandwidth to reach the traffic collector. Recently, In-band Network Telemetry (INT) offered by programmable networks, e.g., P4, has gained attention by providing fine-grained network telemetry. Current attempts on INT are inflexible in collecting telemetry for customs flows according to the desired interval. This letter proposes P4FLOW, a flow monitoring tool for cloud provider networks implemented on programmable data planes. P4FLOW allows the providers to monitor a set of desired flows according to their needs. It reduces at least 1.6x the overhead of telemetry packets compared with the existing approaches.

show abstract

Orchestrating In-Band Data Plane Telemetry With Machine Learning

Cited by 32 publications

References 12 publications

Using Machine Learning and In-band Network Telemetry for Service Metrics Estimation

Using Machine Learning and In-band Network Telemetry for Service Metrics Estimation

Fast and On-demand Network-wide Telemetry via Cluster Based Path Planning

P4Flow: Monitoring Traffic Flows With Programmable Networks

Contact Info

Product

Resources

About