The Performance of Q-Learning within SDN Controlled Static and Dynamic Mesh Networks

Abstract

Current infrastructures are reaching the point where existing networking methods are unable to cope with the exponential growth of traffic and Quality of Service (QoS) requirements. New techniques are necessary to keep pace. One such technique, Software-Defined Networking (SDN) uses a central controller to program many individual network devices. However, SDN uses heuristic algorithms that do not always select the optimal path. This paper looked at creating three Q-Routing algorithms leveraging SDN and Mesh network topologies. Two algorithms used one network metric each (Latency and Bandwidth) and the third used multiple metrics. Results showed that the single metric Q-Routing algorithms on average performed as well as the K-Shortest Path versions while Q-Routing with multiple network metrics failed to match K-Shortest Path (different combination of metrics means these algorithms are not comparable). Results also showed that Q-Routing was able to calculate paths faster than K-Shortest Path in both static and dynamic networks.

Q-Routing Using Multiple QoS Metrics in SDN

“…2. An amalgamated approach where multiple metrics are combined within a single reward formula resulting in the selection of sub-optimal paths [13,17]. 3.…”

“…The existing estimates are combined with the new CEL estimates according to (Fig. 3) using two reward formulas; the original Q-Routing reward formula [10] for the latency part of the estimates and a version of the formula [10] derived to use bandwidth [17] (Eq.2):-…”

Q-Routing with Multiple Soft Requirements

“…where Q is an estimate of the bandwidth of the best path and B(x, n) represents the bandwidth available on the link from x to n [4].…”