Joint Path and Wavelength Selection Using Q-learning in Optical Burst Switching Networks

Abstract

Contention losses which usually do not indicate congestion is a major issue that hinders the deployment of optical burst switching (OBS) networks. Development of efficient path and wavelength selection algorithms is crucial to minimize the burst loss probability (BLP) in OBS networks. In this paper, we handle path selection and wavelength selection in a joint fashion. We formulate the problem of selecting a pair of path and wavelength jointly as a multi-armed bandit problem (MABP) and discuss the difficulties in solving MABP directly. We then rewrite the Q-learning formalism to solve the MABP without explicit model in an online fashion and propose an algorithm to solve the problem near-optimally. The proposed algorithm selects a pair of path and wavelength at each ingress node to minimize the BLP on the long run. Simulation results demonstrate the effectiveness of our algorithm in minimizing the BLP with better link utilization compared to the other proposals in the literature.

Multi-agent Q-learning based Admission Control Mechanism in Heterogeneous Wireless Networks for Multiple Services

“…Q-learning is a self-learning method to optimize the kind of decisions that depend both on the current and the history state-action-pair. Agent in Q-learning system learns how to improve its decisions during the learning process according to its experience [24]. In a decision epoch…”

Reinforcement Learning Based Auction Algorithm for Dynamic Spectrum Access in Cognitive Radio Networks

“…We follow the buffer state model of [8], assuming the number of packets arriving into the buffer during one time slot is a random variable independent of the time t and denoted as j t s A . It follows the Poisson distribution with the average arrival rate A packets per second.…”