Q-Learning for Policy Based SON Management in wireless Access Networks

Abstract

Abstract-Self organized networks has been one of the first concrete implementations of autonomic network management concept. Currently, several Self-Organizing-Network (SON) functions are developed by Radio Access Network (RAN) vendors and already deployed in many networks all around the world. These functions have been designed independently to replace different operational tasks. The concern of making these functions work together in a coherent manner has been studied later in particular in SEMAFOUR project where a Policy Based SON Management (PBSM) framework has been proposed to holistically manage a SON enabled network, namely a network with several individual SON functions. Enriching this PBSM framework with cognition capability is the next step towards the realization of the initial promise of SON concept: a unique self-managed network that responds autonomously and efficiently to the operator high level requirements and objectives. This paper proposes a cognitive PBSM system that enhances the SON management decisions by learning from past experience using Q-learning approach. Our approach is evaluated by simulation on a SON enabled Long-Term Evolution Advanced (LTE-A) network with several SON functions. The paper shows that the decisions are enhanced during the learning process and discusses the implementation options of this solution.

Softwarized and distributed learning for SON management systems

“…In [9] the authors showed that a centralized RL algorithm (Q-learning) is able to learn the optimal policy, satisfying the operator's objectives. However, when the network size, the number of SON functions and instances, as well as the number of possible SCV sets for each SON function grow, the action space that has to be explored by the RL agent explodes, leading to a very slow convergence.…”