Optimizing Handover Parameters by Q-Learning for Heterogeneous Radio-Optical Networks

Abstract

Existing literature studying the access point (AP)-user association problem of heterogeneous radio-optical networks either investigates quasi-static network selection or only considers vertical handover (VHO) dwell time from optical to radio. The quasi-static assumption can result in outdated decisions for highly mobile scenarios. Solely focusing on the optical to radio handover ignores the importance of dwell time for VHO from radio to optical. In this paper, we propose a flexible and holistic framework, that runs a self-optimizing algorithm at the centralized coordinator (CC). This CC resides in the LTE eNodeB and controls the handover parameters of all the visible light communication (VLC) APs under the coverage of the LTE eNodeB. Based on Q-learning approach, the algorithm optimizes the time-to-trigger (T T T) values for VHO between LTE and VLC. Case studies are performed to validate the considerable gain in terms of average throughput by optimizing T T T s. We evaluate the impact of learning parameters on the optimal throughput and convergence speed through trace-driven simulations. The simulation results reveal that the Q-learning based algorithm improves the average throughput of mobile device by 25% when compared to the fixed T T T scheme. Furthermore, this algorithm is capable of self-optimizing handover parameters in an online manner.

Sub-Decimeter VLC 3D Indoor Localization With Handover Probability Analysis

“…The authors in [30] aimed to optimize the waiting time before the vertical handover process based on the rate of blocking and the recovery of VLC optical wireless channels. Authors in [32], [33] employ reinforcement learning to optimize the WT before the handover process. References [34], [35] derived a mathematical expression related to QoS parameters of users in a VLC network.…”

“…Then, Pr{L im,jn ≤ } is re-expressed as (31), where (a) comes from the formula P(X ∩ Y ) = P(X)P(Y ) for two independent random variables X and Y , and u( ) refers to the unit step function. Using the fact that Pr{X ≥ x} = 1 − F X (x) and Pr{a ≤ X ≤ b} = F X (b) − F X (a), the CDF expression of F Li m ,jn ( ) = Pr{L im,jn ≤ )} is re-written as (32). In the next step, we take derivative from the CDF of L im,jn with respect to to compute its pdf, denoted by f Li m ,jn ( ).…”

Advanced Mobility Robustness Optimization Models in Future Mobile Networks Based on Machine Learning Solutions

“…The common types of ML have been implemented to optimize the HCPs of MRO functions, such as the supervised ML [21], [22], [23], [24], [25], [26], [27], unsupervised ML [28], and reinforcement learning [14], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42]. Several ML techniques under each type have been addressed as a solution method.…”

“…The fuzzy Q-learning technique was used throughout numerous works [14], [29], [30], [33]. The research of [31], [32], [35], [38], [39], and [40] presented a Q-learning technique to acquire optimal HCP settings. The Q-learning technique was integrated with other methods such as Q-learning and Analytic hierarchy process technique for order of preference by similarity to ideal solution (AHP-TOPSIS) [34], deep Q-learning [42].…”

Deep Reinforcement Learning-Based Adaptive Handover Mechanism for VLC in a Hybrid 6G Network Architecture

“…In [12], Du et al combined the Q-learning algorithm with transfer learning to improve the efficiency of the algorithm itself, thereby further improving the convergence speed and system performance over those achieved with traditional RL. Shao et al [13] proposed a self-optimization algorithm based on Q-learning, where the switching parameters of the APs were optimized by a centralized coordinator. In addition, other AI algorithms have also been used in VLC handover mechanisms; for example, Ji et al [14] used a support vector machine (SVM) approach and Najla et al [15] used a deep neural network (DNN) approach to propose algorithms that effectively improve the network switching performance in VLC-and RF-based heterogeneous networks.…”