Artificial Intelligence-Based Handoff Management for Dense WLANs: A Deep Reinforcement Learning Approach

Abstract

So far, the handoff management involved in the wireless local area network (WLAN) has mainly fallen into the handoff mechanism and the decision algorithm. The traditional handoff mechanism generates noticeable delays during the handoff process, resulting in discontinuity of service, which is more evident in dense WLANs. Inspired by software-defined networking (SDN), prior works put forward many seamless handoff mechanisms to ensure service continuity. With respect to the handoff decision algorithm, when to trigger handoff and which access point to reconnect to, however, are still tricky problems. In this paper, we first design a self-learning architecture applicable to the SDN-based WLAN frameworks. Along with it, we propose DCRQN, a novel handoff management scheme based on deep reinforcement learning, specifically deep Q-network. The proposed scheme enables the network to learn from actual users' behaviors and network status from scratch, adapting its learning in time-varying dense WLANs. Due to the temporal correlation property, the handoff decision is modeled as the Markov decision process (MDP). In the modeled MDP, the proposed scheme depends on the real-time network statistics at the time of decisions. Moreover, the convolutional neural network and the recurrent neural network are leveraged to extract fine-grained discriminative features. The numerical results through simulation demonstrate that DCRQN can effectively improve the data rate during the handoff process, outperforming the traditional handoff scheme.

Energy Efficient 3-D UAV Control for Persistent Communication Service and Fairness: A Deep Reinforcement Learning Approach

“…In [50], an artificial intelligence framework (AIF) for smart wireless network management was proposed. DCRQN [51] which is a novel Wi-Fi handoff management scheme based on Deep Q-Network (DQN) [52] effectively improves the data rate during the handoff process. In [53], the authors presented DeepNap, which uses a DQN to learn effective BS sleeping policies and reduces the energy consumption of Wi-Fi networks.…”

QAAs: QoS provisioned artificial intelligence framework for AP selection in next-generation wireless networks

“…To be specific, the feature extraction module [24,26] is the key module of the local SDN controller that extracts the spatial as well as temporal features of the raw wireless signal through CNN and RNN respectively. The CNN extracts the relative positions properties of the client in WLAN whereas the deep and wide network measurements are logged on the basis of timestamps that led to modelling of temporal features through RNN.…”

“…Authors in [25], presented a recurrent neural network based handover management scheme which extract the temporal features to make a handover decision in the SDN-based WLAN. Han et al [26] proposed a handoff management scheme based on the Deep Q Network (DQN) that learns the user behaviour and network state from the beginning in order to learn a good handover policy. Unfortunately, the above discussed works fail to guarantee the user QoS requirements in a dense WLAN environment.…”

“…This artificial intelligence framework achieved a significant improvement in throughput, but doesn't consider a holistic framework based on SDN. On the basis of a feature extraction module discussed in [24], authors in [26] proposed a self-learning SDN based artificial intelligence framework to solve the handover management problem in the dense WLAN environment. This DQN based model leverage signal-tointerference-plus-noise ratio (SINR) to define the network state and moreover learns the user behaviour and network status from the beginning without the need of any parameter configuration.…”

Handoff Decisions in Hetnets using Fuzzy Logic and Machine Learning Techniques

“…When dealing with vertical handoffs, there are many challenges. First, because at one time there are more than one devices active, power consumption become a problem [13]. While multiple active network interfaces are running, power drain must be minimized.…”