When Machine Learning Meets Congestion Control: A Survey and Comparison

Abstract

Machine learning (ML) has seen a significant surge and uptake across many diverse applications. The high flexibility, adaptability and computing capabilities it provides extends traditional approaches used in multiple fields including network operation and management. Numerous surveys have explored ML in the context of networking, such as traffic engineering, performance optimization and network security. Many ML approaches focus on clustering, classification, regression and reinforcement learning (RL). The innovation of this research and contribution of this paper lies in the detailed summary and comparison of learning-based congestion control (CC) approaches. Compared with traditional CC algorithms which are typically rule-based, capabilities to learn from historical experience are highly desirable. From the literature, it is observed that RL is a crucial trend among learning-based CC algorithms. In this paper, we explore the performance of RL-based CC algorithms and present current problems with RL-based CC algorithms. We outline challenges and trends related to learning-based CC algorithms.

Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

“…Machine learning (ML) was successfully implemented across disciplines, ranging from healthcare to autonomous driving. Compared to manual tuning methods, ML algorithms are capable of extracting complex patterns from vast amounts of data, learning implicit correlations that enable better generalization and performance [22]. Prior work considered ML-based CC; however, those algorithms often require large memory and computational complexity [22].…”

“…Compared to manual tuning methods, ML algorithms are capable of extracting complex patterns from vast amounts of data, learning implicit correlations that enable better generalization and performance [22]. Prior work considered ML-based CC; however, those algorithms often require large memory and computational complexity [22]. Generally, for CC algorithms to successfully operate, their decision time must be O (RTT).…”

Genetic Improvement of TCP Congestion Avoidance

“…Traditional congestion control algorithms can be divided into two categories: end-to-end and network assisted. While in the former only information about the sender and the receiver is needed, in the latter metrics regarding the network infrastructure are used to take decisions [7].…”

“…In general, if the sender does not receive back the acknowledgment from the receiver after a certain amount of time, the sender may "infer" that the packet is lost. On the other hand, delay-based approaches are better suited for networks that need high speed and flexibility [7], but also in this case calculating the exact transmission delay is tricky; other paths have been researched and some hybrid algorithms have been proposed such as [8].…”