A reinforcement learning-based algorithm for deflection routing in optical burst-switched networks

Abstract

In this paper, we propose a Q-learning based deflection routing algorithm that may be employed to resolve contention in optical burst-switched networks. The main goal of deflection routing is to successfully deflect a burst based only on a limited knowledge that network nodes possess about their environment. Q-learning, one of the reinforcement learning algorithms, has been proposed in the past to help generate deflection decisions. The complexity of existing reinforcement learning-based deflection routing algorithms depends on the number of nodes in the network.The proposed algorithm scales well for larger networks because its complexity depends on the node degree rather than the network size. The algorithm is implemented using the ns-3 network simulator. Simulation results show that it has comparable performance to an existing reinforcement learning deflection routing scheme while having lower memory requirements.

Intelligent Deflection Routing in Buffer-Less Networks

“…Reinforcement learning, which has been implemented in the proposed NN-NDD and ENN-NDD algorithms, provides a systematic framework for processing the gathered information. Various other deflection routing protocols based on reinforcement learning [8]- [10], [30] employ the Q-learning algorithm or its variants.…”

“…The ns-3 [6] implementation of iDef is made publicly available [7]. 2) We introduce the novel Node Degree Dependent (NDD) signaling algorithm [8]. The complexity of the algorithm only depends on the degree of the node that is NDD compliant while the complexity of the other currently available reinforcement learning-based deflection routing algorithms depends on the size of the network.…”

A Predictive Q-Learning Algorithm for Deflection Routing in Buffer-less Networks

“…A drawback of the Q-learning path selection algorithm and RLDRS is that they are not scalable because their complexity depends on the size of the network. The Q-NDD algorithm [11] also employs Q-learning for deflection routing. However, it scales better in larger networks because its complexity depends on the node degree rather than the network size.…”

“…In order to evaluate and compare performance of the proposed PQDR algorithm, we have also implemented in ns-3 the existing RLDRS [9] and Q-NDD algorithm [11]. We compare the algorithms in terms of burst loss probability, number of deflections, average number of hops, and end-to-end delay by simulating the National Science Foundation (NSF) network topology.…”

“…We evaluate performance of the proposed algorithm using the ns-3 [10] network simulator and compare its performance with RLDRS and a newly proposed Node Degree Dependent (Q-NDD) algorithm [11]. This paper is organized as follows.…”