Remote Electrical Tilt Optimization via Safe Reinforcement Learning

Vannella, Filippo; Iakovidis, Grigorios; Håkim, Ezeddin Al; Aumayr, Erik; Feghhi, Saman

doi:10.1109/wcnc49053.2021.9417363

Cited by 25 publications

(22 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, Fig. (16) evaluates the algorithm's stability for increasing level of randomness in the environment's traffic distribution. In order to simulate such randomness, multiple runs over 50 random seeds have been executed.…”

Section: Observed Resultsmentioning

confidence: 99%

“…One way to tackle CCO is to allow coverage regions to change by modifying cells' elevation tilts. Some early decade contributions made use of research operations methods [12,13] to tune antennas parameters, while most recent years, motivated by the late success of Deep Learning (DL) and Reinforcement Learning (RL), have seen the rise of solutions employing DRL algorithms [14][15][16][17]. RL, indeed, seems to be a valuable tool for CCO since it can learn and adapt to the dynamics of the environment [14].…”

Section: A State Of the Artmentioning

confidence: 99%

“…In [17], a solution for optimizing the antenna parameters of a HetNet is proposed addressing the unstable convergence of hyperparameters. Finally, [16] proposes a safe antenna tilt update policy to avoid the execution of degrading actions and improve system's reliability. All of the aforementioned papers are innovative, as they respectively propose solutions to improve scalability [14,17], sampleefficiency [15] and reliability [16].…”

Section: A State Of the Artmentioning

confidence: 99%

“…Finally, [16] proposes a safe antenna tilt update policy to avoid the execution of degrading actions and improve system's reliability. All of the aforementioned papers are innovative, as they respectively propose solutions to improve scalability [14,17], sampleefficiency [15] and reliability [16]. Nevertheless, none of them makes use of realistic data measurements like MDT for network state representation and training.…”

Section: A State Of the Artmentioning

confidence: 99%

“…However, this is generally very inconvenient due to slow training times and potential degrading actions performed during the exploration phase of the training. In this regard, many contributions in literature focused on risk sensitivity and damage avoidance during the exploration phase of an agent under the mantle of safe RL [16,35].…”

Section: System Modelmentioning

confidence: 99%

See 4 more Smart Citations

Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning

Skocaj¹,

Amorosa²,

Ghinamo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent years witnessed a remarkable increase in the availability of data and computing resources in communication networks. This contributed to the rise of data-driven over model-driven algorithms for network automation. This paper investigates a Minimization of Drive Tests (MDT)-driven Deep Reinforcement Learning (DRL) algorithm to optimize coverage and capacity by tuning antennas tilts on a cluster of cells from TIM's cellular network. We jointly utilize MDT data, electromagnetic simulations, and network Key Performance indicators (KPIs) to define a simulated network environment for the training of a Deep Q-Network (DQN) agent. Some tweaks have been introduced to the classical DQN formulation to improve the agent's sample efficiency, stability and performance. In particular, a custom exploration policy is designed to introduce soft constraints at training time. Results show that the proposed algorithm outperforms baseline approaches like DQN and best-first search in terms of long-term reward and sample efficiency. Our results indicate that MDT-driven approaches constitute a valuable tool for autonomous coverage and capacity optimization of mobile radio networks.

show abstract

Section: Observed Resultsmentioning

confidence: 99%

Section: A State Of the Artmentioning

confidence: 99%

Section: A State Of the Artmentioning

confidence: 99%

Section: A State Of the Artmentioning

confidence: 99%

Section: System Modelmentioning

confidence: 99%

See 3 more Smart Citations

Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning

Skocaj¹,

Amorosa²,

Ghinamo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Evaluation of Intrinsic Explainable Reinforcement Learning in Remote Electrical Tilt Optimization

Ruggeri,

Terra,

Inam

et al. 2023

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Decomposition methods with deep corrections for reinforcement learning

Bouton

Julian

Nakhaei

et al. 2019

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

Decomposition methods have been proposed to approximate solutions to large sequential decision making problems. In contexts where an agent interacts with multiple entities, utility decomposition can be used to separate the global objective into local tasks considering each individual entity independently. An arbitrator is then responsible for combining the individual utilities and selecting an action in real time to solve the global problem. Although these techniques can perform well empirically, they rely on strong assumptions of independence between the local tasks and sacrifice the optimality of the global solution. This paper proposes an approach that improves upon such approximate solutions by learning a correction term represented by a neural network. We demonstrate this approach on a fisheries management problem where multiple boats must coordinate to maximize their catch over time as well as on a pedestrian avoidance problem for autonomous driving. In each problem, decomposition methods can scale to multiple boats or pedestrians by using strategies involving one entity. We verify empirically that the proposed correction method significantly improves the decomposition method and outperforms a policy trained on the full scale problem without utility decomposition. decomposition methods with deep corrections for reinforcement learning 2 bined in real-time through an arbitrator that maximizes the expected utility [22]. Applications to aircraft collision avoidance with multiple intruders explore summing or using the minimum state-action values [7], [20]. While these approaches tend to perform well empirically, they sacrifice optimality. The solution to each subproblem assumes that its individual policy will be followed regardless of other entities, in contrast to the global policy considering all subproblems [23].Previous approaches to scale decision algorithms using decomposition methods relied on a distributed agent architecture [22]. Each agent is responsible for addressing one of the multiple objectives required to achieve a complex task. At each time step an arbitrator must decide between the different actions recommended by these agents and address possible conflicts. Possible arbitration strategies are command fusion [7], voting [22], lexicographic ordering [33] or utility fusion [22], [23]. It has been shown that utility fusion offers a more principled way of deciding between the individual agents compared to a voting-based approach or command fusion [22]. An alternative approach in leader follower scenarios consists of reducing the problem of controlling the group of agent to controlling a single group leader [15]. Other approaches rely on distributed learning algorithms, such as independent Q-learning, where each agent learns a policy without being aware of the other agents actions [28]. The underlying assumption of independence between agents in these algorithms trades off the benefit of collaboration for an easier learning of the task [8].Utility decomposition methods have been used in many practic...

show abstract

Remote Electrical Tilt Optimization via Safe Reinforcement Learning

Cited by 25 publications

References 6 publications

Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning

Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning

Evaluation of Intrinsic Explainable Reinforcement Learning in Remote Electrical Tilt Optimization

Decomposition methods with deep corrections for reinforcement learning

Contact Info

Product

Resources

About