Joint Optimization of Energy Efficiency and User Outage Using Multi-Agent Reinforcement Learning in Ultra-Dense Small Cell Networks

Kuadey

et al. 2023

JDSIS

Studies on developing future-generation wireless systems are expected to support increased infrastructure development and device subscriptions with densely deployed base stations (BSs). Economically, decreasing BS energy consumption levels and achieving "greenness" remain key factors for the giant industry. Some research works have proposed deep reinforcement techniques to solve energy management (EM) issues in cellular networks. However, these techniques are inefficient in a distributive network environment and expose the devices to privacy issues. Federated learning (FL) is proven to enforce device privacy and train models distributively. Thus, this work proposes an autonomous switching mode framework for BSs based on federated-deep reinforcement learning to address the aforementioned challenges encountered by prior studies. Specifically, we deploy multiple deep reinforcement learning (mDRL) agents to influence the decision of the BS for energy management (EM). On the other hand, to make deep reinforcement learning (DRL)-based decisions feasible and satisfy D-QoS, we train the DRL-agents distributively by employing the federated learning (FL) concept. The results show the effectiveness of our proposed framework under distributed network scenarios compared with other benchmark algorithms.

Section: Knowledge Workmentioning

confidence: 99%

Federated-Based Deep Reinforcement Learning (Fed-DRL) for Energy Management in a Distributive Wireless Network

Kuadey

et al. 2023

JDSIS

“…Multi-Agent Reinforcement Learning. [5] EE and SE for UAV-CRN. Delay aware-approximate optimization strategy.…”

Section: Research Topic On Uav-based Sensing Modeling Approach Refere...mentioning

confidence: 99%

“…As a result, various applications like military, telecommunication, surveillance, medical supplies delivery, and rescue operations weigh the UAVs as perfect enablers for seamless communications [4]. The UAVs are also expected to be the center of 6G wireless networks, which have gained significant attention from the research community and industry to support emerging mobile services [5].…”

Section: Introductionmentioning

confidence: 99%

UAV aided virtual cooperative spectrum sensing for cognitive radio networks

Gul,

Kim,

Ali

et al. 2023

PLoS ONE

Cooperative spectrum sensing (CSS) involves multiple secondary users (SUs) reporting primary user (PU) channel sensing states to the fusion center (FC). However, the high overheads associated with multi-user CSS impose power limitations that limit its usefulness in unmanned aerial vehicle (UAV) networks. To address this challenge, we propose a virtual CSS, where a single UAV conducts CSS while following a circular flight trajectory in the air. The novelty of our approach is presenting a working frame structure for the UAV flight, including sensing and data transmission periods with further division of the sensing time into mini-sensing slots. In the virtual CSS, UAV performs local sensing decisions in each mini-slot and accumulates them for a final decision. The proposed virtual CSS scheme exploits sequential decision fusion (SDF), which sequentially adds individual mini-slot decisions. Additionally, we leverage machine learning (ML), employing AdaBoost ensembling classifier (ENC), to inspect flight conditions and reconfigure mini-slot periods dynamically for both traditional decision fusion (TDF) and our proposed SDF schemes. Furthermore, we identify an optimal decision threshold (ODT) for the proposed SDF, enabling the comparison of sequential results with an adjustable threshold through majority voting. This novel approach results in energy efficiency and improved throughput for virtual CSS using SDF, surpassing the performance of TDF, which relies on collecting entire mini-slot reports for its final decision. Simulation results demonstrate the effectiveness of the proposed SDF following the ENCODT (SDF-ENCODT) scheme compared to existing techniques from the literature. We explore varying levels of UAV flight velocities, moving radius, detection probability demand, and channel signal-to-noise ratio (SNR), reinforcing the significance of our contribution. Our research highlights the motivation to address spectrum scarcity in UAV communication by proposing an innovative virtual CSS scheme based on SDF. The proposed approach enhances spectrum utilization, overcomes power limitations, and substantially improves CSS for UAV networks.

“…A double-DQN-based solution for resource allocation was given in [19] to maximize the energy efficiency. Furthermore, the authors in [20] proposed a multi-agent distributed Q-learning-based algorithm to optimize energy efficiency and user outage. In [21], long short-term memory (LSTM) was exploited to control the BS on/off decision.…”

Section: Related Workmentioning

confidence: 99%

Carbon-Neutral Cellular Network Operation Based on Deep Reinforcement Learning

Kim

2022

Energies

With the exponential growth of traffic demand, ultra-dense networks have been proposed to cope with such demand. However, the increase of the network density causes more power use, and carbon neutrality becomes an important concept to decrease the emission and production of carbon. In cellular networks, emission and production can be directly related to power consumption. In this paper, we aim to achieve carbon neutrality, as well as maximize network capacity with given power constraints. We assume that base stations have their own renewable energy sources to generate power. For carbon neutrality, we control the power consumption for base stations by adjusting the transmission power and switching off base stations to balance the generated power. Given such power constraints, our goal is to maximize the network capacity or the rate achievable for the users. To this end, we carefully design the objective function and then propose an efficient Deep Deterministic Policy Gradient (DDPG) algorithm to maximize the objective. A simulation is conducted to validate the benefits of the proposed method. Extensive simulations show that the proposed method can achieve carbon neutrality and provide a better rate than other baseline schemes. Specifically, up to a 63% gain in the reward value was observed in the DDPG algorithm compared to other baseline schemes.