ReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning

Zhang, Han; Li, Wenzhong; Gao, Shaohua; Wang, Xiaoliang; Ye, Baoliu

doi:10.1109/infocom.2019.8737649

Cited by 88 publications

(31 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies applied AI technology to a multipath TCP scheduler to refine the scheduling policy dynamically in accordance with its surrounding environments. In [ 80 ], Zhang et al proposed a RL-based scheduler, namely, ReLes, to generate packet scheduling policy dynamically. Different from the traditional pre-defined fixed policies, ReLes is able to adjust the scheduling policy dynamically based on network environments.…”

Section: Congestion Control and Schedulermentioning

confidence: 99%

A Brief Review of Multipath TCP for Vehicular Networks

Chao

Yoshinaga

et al. 2021

Sensors

View full text Add to dashboard Cite

Multipath TCP (MPTCP) is one of the most important extensions to TCP that enables the use of multiple paths in data transmissions for a TCP connection. In MPTCP, the end hosts transmit data across a number of TCP subflows simultaneously on one connection. MPTCP can sufficiently utilize the bandwidth resources to improve the transmission efficiency while providing TCP fairness to other TCP connections. Meanwhile, it also offers resilience due to multipath data transfers. MPTCP attracts tremendous attention from the academic and industry field due to the explosive data growth in recent times and limited network bandwidth for each single available communication interface. The vehicular Internet-of-Things systems, such as cooperative autonomous driving, require reliable high speed data transmission and robustness. MPTCP could be a promising approach to solve these challenges. In this paper, we first conduct a brief survey of existing MPTCP studies and give a brief overview to multipath routing. Then we discuss the significance technical challenges in applying MPTCP for vehicular networks and point out future research directions.

show abstract

Section: Congestion Control and Schedulermentioning

confidence: 99%

A Brief Review of Multipath TCP for Vehicular Networks

Chao

Yoshinaga

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…On the one hand, recent work has initiated the modeling of MP scheduling as a decision-making problem, ultimately leveraging RL-based solutions. In particular, reinforcement learning based scheduler (ReLeS) in [272] adopts DQN in order to find optimal scheduling policies; Peekaboo [270] models the scheduling task as a contextual MAB and employs LinUCB and a stochastic adjustment in order to derive a probabilistic policy. Modified-Peekaboo is proposed in [273], in order to better deal with the high dynamicity of 5G mmWave access.…”

Section: Open Challengesmentioning

confidence: 99%

User-Centric Radio Access Technology Selection: A Survey of Game Theory Models and Multi-Agent Learning Algorithms

et al. 2021

View full text Add to dashboard Cite

User-centric radio access technology (RAT) selection is a key communication paradigm, given the increased number of available RATs and increased cognitive capabilities at the user end. When considered against traditional network-centric approaches, user-centric RAT selection results in reduced network-side management load, and leads to lower operational costs for RATs, as well as improved quality of service (QoS) and quality of experience (QoE) for users. The complex between-users interactions involved in RAT selection require, however, specific analyses, toward developing reliable and efficient schemes. Two theoretical frameworks are most often applied to user-centric RAT selection analysis, i.e., game theory (GT) and multi-agent learning (MAL). As a consequence, several GT models and MAL algorithms have been recently proposed to solve the problem at hand. A comprehensive discussion of such models and algorithms is, however, currently missing. Moreover, novel issues introduced by next-generation communication systems also need to be addressed. This paper proposes to fill the above gaps by providing a unified reference for both ongoing research and future research directions in the field. In particular, the review addresses the most common GT and MAL models and algorithms, and scenario settings adopted in user-centric RAT selection in terms of utility function and network topology. Regarding GT, the review focuses on non-cooperative models, because of their widespread use in RAT selection; as for MAL, a large number of algorithms are described, ranging from game-theoretic to reinforcement learning (RL) schemes, and also including most recent approaches, such as deep RL (DRL) and multi-armed bandit (MAB). Models and algorithms are analyzed by comparatively reviewing relevant literature. Finally, open challenges are discussed, in light of ongoing research and standardization activities.

show abstract

“…Zhang et al [52] proposed ReLeS,a Reinforcement Learning based Scheduler for MPTCP, which applies deep reinforcement learning (DRL) techniques to learn a neural network (NN) to find the best MPTCP distribution policy. The employed reward function is complex as it determines multi QoS features.…”

Section: Learning-based Schedulersmentioning

confidence: 99%

“…Objective Lim et al [24] Heuristics Earliest Completion First Pokhrel et al [31] Heuristics throughput-based Chung et al [10] ML predictive QoS Beig et al [7] Heuristics throughput-based Pokhrel and Mandjes [37] Q-Learning QoS Chiariotti et al [9] Heuristics QoS Zhang et al [52] RL QoS Wu et al [49] RL highest throughput Pokhrel and Williamson [42] game theory throughput and responsiveness Huang et al [18] A distributed (DRL) loss and delay Pokhrel and Garg [35] DRL QoS Abbasloo et al [1] DRL delays Pokhrel and Choi [30] Federated learning (BFL) privacy-based Table 3.1 provides general comparison between some of heuristics and learning-based MPTCP scheduling techniques.…”

Section: Paper Techniquementioning

confidence: 99%

A Survey of Multipath TCP Scheduling Schemes: Open Challenges and Potential Enablers

Asiri¹

2021

Preprint

View full text Add to dashboard Cite

Today, mobile devices like smartphones are supported with various wireless radio interfaces including cellular (3G/4G/LTE) and Wi-Fi (IEEE 802.11) [42]. The legacy devices can only communicate with only one interface. The Transmission Control Protocol, or TCP, has a limitation inability to change connection settings without breaking the connection. In this paper, we explain how multi-path TCP (MPTCP) protocol has been proposed to solve TCP single-interface limitation and provides a huge improvement on application performance by using multiple paths transparently (auto path changing). We discuss the last mile, which is the final networking segment that carried all network traffic. Indeed, the available bandwidth in last-mile link can effectively harm the network throughput as it limits the amount of transmitted data. We found that the quality of the last mile networks significantly determines the reliability and quality of the carrying network. We believe MPTCP can provide a convenient solution for the last mile problem. We provide a holistic view of the challenges and potential enablers in details.<br>

show abstract

ReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning

Cited by 88 publications

References 21 publications

A Brief Review of Multipath TCP for Vehicular Networks

A Brief Review of Multipath TCP for Vehicular Networks

User-Centric Radio Access Technology Selection: A Survey of Game Theory Models and Multi-Agent Learning Algorithms

A Survey of Multipath TCP Scheduling Schemes: Open Challenges and Potential Enablers

Contact Info

Product

Resources

About