Enhancing Handover for 5G mmWave Mobile Networks Using Jump Markov Linear System and Deep Reinforcement Learning

Abstract

The Fifth Generation (5G) mobile networks use millimeter waves (mmWaves) to offer gigabit data rates. However, unlike microwaves, mmWave links are prone to user and topographic dynamics. They easily get blocked and end up forming irregular cell patterns for 5G. This in turn causes too early, too late, or wrong handoffs (HOs). To mitigate HO challenges, sustain connectivity, and avert unnecessary HO, we propose an HO scheme based on a jump Markov linear system (JMLS) and deep reinforcement learning (DRL). JMLS is widely known to account for abrupt changes in system dynamics. DRL likewise emerges as an artificial intelligence technique for learning highly dimensional and time-varying behaviors. We combine the two techniques to account for time-varying, abrupt, and irregular changes in mmWave link behavior by predicting likely deterioration patterns of target links. The prediction is optimized by meta training techniques that also reduce training sample size. Thus, the JMLS–DRL platform formulates intelligent and versatile HO policies for 5G. When compared to a signal and interference noise ratio (SINR) and DRL-based HO scheme, our HO scheme becomes more reliable in selecting reliable target links. In particular, our proposed scheme is able to reduce wasteful HO to less than 5% within 200 training episodes compared to the DRL-based HO scheme that needs more than 200 training episodes to get to less than 5%. It supports longer dew time between HOs and high sum rates by ably averting unnecessary HOs with almost half the HOs compared to a DRL-based HO scheme.

Pattern Based Mobility Management in 5G Networks With a Game Theoretic-Jump Markov Linear System Approach

“…Considering multiple coexisting and competing mmWave link deteriorating factors as described earlier, various HO schemes based on tradeoffs and machine learning techniques including GT and artificial Intelligence (AI) have been proposed for 5G mobile networks [6]. In [7], for instance, a tradeoff between high mmWave Base Station (mm-BS) spatial density is done against limited front and backhaul capacity.…”

“…Due to limited and sometimes fast changing channel state information (CSI), the initial values of the deterioration pattern are inferred using Expected Maximization (EM) algorithm. However, EM is intractable in instances where CSI changes are drastic [6]. Thus, it is important to understand the rapid fluctuation and intermittent connectivity in mmWave networks [4].…”

“…Our proposed HO scheme ultimately selects target links that sustain mmWave connectivity longer than the rest. JMLS modelling is known to account for both gradual and abrupt changes [6]. We use this JMLS capability in the 5G network to learn to predict the mmWave target link's abrupt and gradual deterioration pattern and vulnerability prior to executing a HO.…”

Milimeter-Wave Massive Mimo: A Promising Technology for New and Emerging Networks

“…Unwanted HO initiations results in unstable network performances. Integration of jump Markov linear system (JMLS) and deep reinforcement learning (DRL) is applied in [81] to avoid the execution of undesirable HO. The scheme is updated at an interval using DRL and meta training procedures.…”

“…Meta training is a strategy that leverages existing training data with similar characteristics to inform new decision-making processes. At full update, the reliance on acquiring entirely new training datasets for making decision in a new location is minimized [81]. A data-driven HO estimation scheme using recurrent deep learning (RDL) design in [83] achieved improved user experience, low latency, minimal signaling overhead, and reduced resource wastage.…”