RL-Routing: An SDN Routing Algorithm Based on Deep Reinforcement Learning

Chen, Yiren; Rezapour, Amir; Tzeng, Wen-Guey; Tsai, Shi-Chun

doi:10.1109/tnse.2020.3017751

Cited by 104 publications

(35 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Fu et al [173], because they particularly focus on a type of scenario (data center networks) and carefully design their approach around it. • Chen et al [177], as their implementation and evaluation is very complete, and close to real scenarios. Therefore, we recommend to follow the work from these research teams in case of interest in the field.…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 85%

See 1 more Smart Citation

A Survey on Machine Learning Techniques for Routing Optimization in SDN

et al. 2021

View full text Add to dashboard Cite

In conventional networks, there was a tight bond between the control plane and the data plane. The introduction of Software-Defined Networking (SDN) separated these planes, and provided additional features and tools to solve some of the problems of traditional network (i.e., latency, consistency, efficiency). SDN is a flexible networking paradigm that boosts network control, programmability and automation. It proffers many benefits in many areas, including routing. More specifically, for efficiently organizing, managing and optimizing routing in networks, some intelligence is required, and SDN offers the possibility to easily integrate it. To this purpose, many researchers implemented different machine learning (ML) techniques to enhance SDN routing applications. This article surveys the use of ML techniques for routing optimization in SDN based on three core categories (i.e. supervised learning, unsupervised learning, and reinforcement learning). The main contributions of this survey are threefold. Firstly, it presents detailed summary tables related to these studies and their comparison is also discussed, including a summary of the best works according to our analysis. Secondly, it summarizes the main findings, best works and missing aspects, and it includes a quick guideline to choose the best ML technique in this field (based on available resources and objectives). Finally, it provides specific future research directions divided into six sections to conclude the survey. Our conclusion is that there is a huge trend to use intelligence-based routing in programmable networks, particularly during the last three years, but a lot of effort is still required to achieve comprehensive comparisons and synergies of approaches, meaningful evaluations based on open datasets and topologies, and detailed practical implementations (following recent standards) that could be adopted by industry. In summary, future efforts should be focused on reproducible research rather than on new isolated ideas. Otherwise, most of these applications will be barely implemented in practice.

show abstract

Section: ) Deep Reinforcement Learningmentioning

confidence: 85%

“…Chen et al [177] comprehensively analyze the need for optimized routing in SDN and present RL-Routing. After an extensive evaluation based on a real SDN controller and networks, RL-Routing proves to offer better results than other routing algorithms like OSPF and Least Loaded (LL).…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 99%

A Survey on Machine Learning Techniques for Routing Optimization in SDN

et al. 2021

View full text Add to dashboard Cite

show abstract

“…They designed a reward function to simultaneously optimize the deadline meet rate and flow completion time (FCT) metrics respectively accounted for mice and elephant flows. Chen et al 34 proposed a reinforcement routing algorithm named RL‐Routing for SDN networks to optimize delay and throughput. They designed RL‐Routing based on the dueling double DQN algorithm.…”

Section: Related Workmentioning

confidence: 99%

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks

Rikhtegar

Bushehrian

Keshtgari

2021

Int J Communication

View full text Add to dashboard Cite

Summary Data center networks (DCNs) are facing challenging control problems such as flow scheduling, congestion control, load balancing, and bandwidth allocation when dynamically handling heterogeneous mice and elephant flows. This paper focuses on the load balancing in DCNs as an online decision‐making problem with high complexity (NP‐hard problem). Traditional load balancing approaches are usually the heuristic algorithms relying on the operator's viewpoint and a short‐term knowledge of the traffic conditions and network environment. This paper proposes DeepRLB, a deep reinforcement learning (DRL)‐based load balancing approach for software‐defined networking‐based DCNs which exploits the deep deterministic policy gradient (DDPG) algorithm to adaptively learn the link‐weight values by observing the traffic flow characteristics. The actor and critic neural networks in the DDPG model have been designed based on two learning models: fully connected (DeepRLB‐Full) and convolutional (DeepRLB‐Conv). The output of the model is further used by the controller to determine the forwarding paths for traffic flows accordingly. We evaluated the performance of DeepRLB by comparing it with ECMP as well as a per‐packet load balancing algorithm in three different data center topologies under 30 heterogeneous traffic demand matrices. The obtained results showed that the DeepRLB algorithm outperforms the ECMP with respect to studied load balancing metrics considerably (DeepRLB‐Conv achieved the best performance). As a sample, DeepRLB‐Full and DeepRLB‐Conv were able to reach 19.14% and 24.46% reduction in the total number of over‐utilized and under‐utilized network links compared to ECMP, respectively.

show abstract

“…A reward is used to reroute the critical flows to balance the link utilization and set up the flows to reflect the network performance. In [39], the authors also proposed positive and negative compensation values using network throughput and delay to optimize the reward for network throughput. After proper learning, the agent proposes dueling double-deep Q-learning-based RL-routing, which predicts the future behavior of the underlying network and suggests better routing paths between switches.…”

Section: Reinforcement-learning-based Network Routingmentioning

confidence: 99%

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems

Shin

Kim

2021

Applied Sciences

View full text Add to dashboard Cite

Research has been conducted to efficiently transfer blocks and reduce network costs when decoding and recovering data from an erasure coding-based distributed file system. Technologies using software-defined network (SDN) controllers can collect and more efficiently manage network data. However, the bandwidth depends dynamically on the number of data transmitted on the network, and the data transfer time is inefficient owing to the longer latency of existing routing paths when nodes and switches fail. We propose deep Q-network erasure coding (DQN-EC) to solve routing problems by converging erasure coding with DQN to learn dynamically changing network elements. Using the SDN controller, DQN-EC collects the status, number, and block size of nodes possessing stored blocks during erasure coding. The fat-tree network topology used for experimental evaluation collects elements of typical network packets, the bandwidth of the nodes and switches, and other information. The data collected undergo deep reinforcement learning to avoid node and switch failures and provide optimized routing paths by selecting switches that efficiently conduct block transfers. DQN-EC achieves a 2.5-times-faster block transmission time and 0.4-times-higher network throughput than open shortest path first (OSPF) routing algorithms. The bottleneck bandwidth and transmission link cost can be reduced, improving the recovery time approximately twofold.

show abstract

RL-Routing: An SDN Routing Algorithm Based on Deep Reinforcement Learning

Cited by 104 publications

References 26 publications

A Survey on Machine Learning Techniques for Routing Optimization in SDN

A Survey on Machine Learning Techniques for Routing Optimization in SDN

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems

Contact Info

Product

Resources

About