A Reinforcement Learning Approach to Congestion Control of High-Speed Multimedia Networks

Shaio, Ming-Chang; Tan, Shun-Wen; Hwang, Kao-Shing; Wu, Cheng-Shong

doi:10.1080/01969720590897224

Cited by 22 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been several efforts that applied RL to congestion control in specialized domains. Prior work in [52] employed RL to create a cooperative congestion control controller for multimedia networks. RL has also been used to solve congestion problems in wireless sensor networks [67].…”

Section: Related Workmentioning

confidence: 99%

Towards a Learning-Based Framework for Self-Driving Design of Networking Protocols

Pasandi

Nadeem

2021

IEEE Access

View full text Add to dashboard Cite

Networking protocols are designed through long-standing and hard-working human efforts. Machine Learning (ML)-based solutions for communication protocol design have been developed to avoid manual effort to adjust individual protocol parameters. While other proposed ML-based methods focus mainly on tuning individual protocol parameters (e.g. contention window adjustment), our main contribution is to propose a new Deep Reinforcement Learning (DRL) framework to systematically design and evaluate networking protocols. We decouple the protocol into a set of parametric modules, each representing the main protocol functionality that is used as a DRL input to better understand and systematically analyze the optimization of generated protocols. As a case study, we introduce and evaluate DeepMAC a framework in which the MAC protocol is decoupled into a set of blocks across popular 802.11 WLANs (e.g. 802.11 a/b/g/n/ac). We are interested to see which blocks are selected by DeepMAC across different networking scenarios and whether DeepMAC is capable of adapting to network dynamics.INDEX TERMS Communication protocols, deep learning, machine-generated algorithm, protocol design, reinforcement learning.This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2021.3061729, IEEE Access H. B. Pasandi, T. Nadeem: Towards a Learning-Based Framework for Self-Driving Design of Networking Protocols II. BACKGROUND HANNANEH BARAHOUEI PASANDI (Student Member, IEEE) received the master's degree from Université Grenoble Alpes. She is currently pursuing the Ph.D. degree with the Department of Computer Science, Virginia Commonwealth University. Her research interests include wireless communication, with an emphasis on the design and implementation of data-driven MAC protocols that not only consider conventional factors but also extract and include important environmental factors in designing protocols for IEEE WLANs.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards a Learning-Based Framework for Self-Driving Design of Networking Protocols

Pasandi

Nadeem

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The authors in Reference 21 proposed RL congestion controller (RLCC) for Asynchronous Transfer Mode (ATM) networks. RLCC receives signals generated by a reward evaluator and takes the best action to control source flow to gain high throughput and low cell loss rate.…”

Section: Related Workmentioning

confidence: 99%

LearningCC: An online learning approach for congestion control

Zhang

Lei

Zhang

et al. 2021

Trans Emerging Tel Tech

View full text Add to dashboard Cite

It is noticed that traditional loss based algorithms are not moving data very well in today's Internet. To get bottleneck bandwidth fully utilized, these algorithms will keep increasing the congestion window and maintain a standing queue of packets in routers, which cause large end to end latency. Such long latency is quite annoying to these ever growing delay sensitive applications. Recently, much effort has been devoted by researchers from both academia and industry to design better congestion control protocols and make a better Internet. In this article, we present an online learning based approach named LearningCC for congestion control. Instead of adjusting the congestion window with fixed rule as these traditional algorithms do, there are several options for an endpoint to choose. Each option is mapped as an arm of a bandit machine. The exploration and exploitation scheme is used to guide the selection on congestion window update rule. Through trial and error, an endpoint improves its performance by choosing action with better reward. Experiments are conducted on ns3 platform to verify the effectiveness of LearningCC. Results indicate it achieves lower transmission delay than loss based algorithms. Especially, we found LearningCC makes significant throughput improvement in link suffering from random loss.

show abstract

“…Second, mobile devices such as smart phones, often connect to wireless networks including WIFI and 4G cellular in an ad hoc fashion. As such, more flexible network topologies and diversified flows are a major challenge [77]. Traditional ML approaches are not dynamic enough to cope with diverse network environments based on trained models, unlike RL algorithms.…”

Section: Rl-based Congestion Control Algorithmsmentioning

confidence: 99%

“…For instance, [84], [85], [36] and [32] propose an AC algorithm to deal with congestion problems in networks with time-varying flows. In [77], the RLbased CC algorithms are used in networks with sparse rewards such as video games, while in [86], the scenario focuses on continuous, large state-action spaces.…”

Section: A Window Updating In End-to-end Networkmentioning

confidence: 99%

When Machine Learning Meets Congestion Control: A Survey and Comparison

Jiang

et al. 2020

Preprint

View full text Add to dashboard Cite

Machine learning (ML) has seen a significant surge and uptake across many diverse applications. The high flexibility, adaptability and computing capabilities it provides extends traditional approaches used in multiple fields including network operation and management. Numerous surveys have explored ML in the context of networking, such as traffic engineering, performance optimization and network security. Many ML approaches focus on clustering, classification, regression and reinforcement learning (RL). The innovation of this research and contribution of this paper lies in the detailed summary and comparison of learning-based congestion control (CC) approaches. Compared with traditional CC algorithms which are typically rule-based, capabilities to learn from historical experience are highly desirable. From the literature, it is observed that RL is a crucial trend among learning-based CC algorithms. In this paper, we explore the performance of RL-based CC algorithms and present current problems with RL-based CC algorithms. We outline challenges and trends related to learning-based CC algorithms.

show abstract

A Reinforcement Learning Approach to Congestion Control of High-Speed Multimedia Networks

Cited by 22 publications

References 12 publications

Towards a Learning-Based Framework for Self-Driving Design of Networking Protocols

Towards a Learning-Based Framework for Self-Driving Design of Networking Protocols

LearningCC: An online learning approach for congestion control

When Machine Learning Meets Congestion Control: A Survey and Comparison

Contact Info

Product

Resources

About