Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous Multi-RAT Networks

Allahham, Mhd Saria; Abdellatif, Alaa Awad; Mhaisen, Naram; Mohamed, Amr; Erbad, Aiman; Guizani, Mohsen

doi:10.1109/tccn.2022.3155727

Cited by 23 publications

(5 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this subsection, we allocate power for beamforming to serve the users by activating the minimal number of grids from the HGA, guided by the information obtained from the sensing process. For our scenario, traditional optimization solution is not likely appropriate to solve the communication resource allocation management policy as we consider the dynamical environment [44]. Therefore, the DRL model can be employed for the power allocation management policy that utilizes dynamic programming strategy for adapting to the dynamic environment and learning the optimal and practical strategies to solve the problem through trial and error technique [45], [46].…”

Section: B Power Allocation Based On Drlmentioning

confidence: 99%

Age of Sensing Empowered Holographic ISAC Framework for NextG Wireless Networks: A VAE and DRL Approach

Adhikary,

Raha,

Qiao

et al. 2024

Preprint

View full text Add to dashboard Cite

This paper proposes an artificial intelligence (AI) framework that leverages integrated sensing and communication (ISAC), aided by the age of sensing (AoS) to ensure the timely location updates of the users for a holographic MIMO (HMIMO)- enabled wireless network. The AI-driven framework guarantees optimal power allocation for efficient beamforming by activating the minimal number of grids from the HMIMO base station. An optimization problem is formulated to maximize the sensing utility function, aiming to maximize the signal-to-interference-plus-noise ratio (SINR) of the received signal, beam-pattern gains to improve the sensing SINR of reflected echo signals and maximizing the evidence lower bound minus loss function, which in turn minimizes the losses of the ISAC process, and maximizes achievable rate for efficient power allocation. A novel AI-driven framework is presented to tackle the formulated NP-hard problem by decomposing it into two problems: a sensing problem and a power allocation problem. The sensing problem is solved by employing a variational autoencoder (VAE)-based mechanism that obtains the sensing information leveraging AoS, which is used for the location update. Subsequently, a deep deterministic policy gradient-based deep reinforcement learning scheme is devised to allocate the desired power by activating the required grids based on the findings achieved with the VAE-based mechanism. Simulation results demonstrate the superior performance of the proposed AI framework compared to advantage actor-critic and deep Q-network-based methods, achieving a cumulative average SINR improvement of 8.5 dB and 10.27 dB, and a cumulative average achievable rate improvement of 21.59 bps/Hz and 4.22 bps/Hz, respectively.

show abstract

Section: B Power Allocation Based On Drlmentioning

confidence: 99%

Age of Sensing Empowered Holographic ISAC Framework for NextG Wireless Networks: A VAE and DRL Approach

Adhikary,

Raha,

Qiao

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The Non-Linear Auto-Regressive External/Exogenous (NARX)-based ANN aims to minimize the rate of sending SRS and achieves an improved accuracy. A distributed framework for dynamic network selection was proposed considering dynamic parameters such as user mobility and battery life [24]. A deep Multi-Agent Reinforcement Learning (DMARL) algorithm, was proposed to optimize cost and energy-e cient manner.…”

Section: Background and Related Workmentioning

confidence: 99%

Switching of Interfaces and Self-optimization of weights using Backpropagation ANN in WHN Enviornment

Rani

2023

Preprint

View full text Add to dashboard Cite

In today’s scenario, mobile communication is facing a healthy competition due to different networks, interfaces, channels, and many more available in wireless heterogeneous environment. The problem arises when customers/users get the availability of many interfaces at the same time. At that time users need an intelligent or smart mechanism to connect them to the best services according to their requirements/preferences. Interface management manages available interfaces and connects the user with the best. In this paper, Interface management with Artificial Neural Network (ANN) allows the smart use of different radio accesses/interfaces. The selection is made with different parameters of different networks. This paper proposed a backpropagation neural network that is used for the switching in between different networks-3G, WLAN, 4G and 5G. The different parameters of a network are used as the selection parameters with assigning proper weights. Weights are initialized with fuzzy AHP and optimized with Back Propagation Neural Network (BPNN). The target value and the actual value is compared and their difference used as the adjusting value for the weights to get the optimum value. The backpropagation is used to train the network. The comparison among the projected algorithm and the existing algorithm shows the valuablity of the new method and the best connectivity of the network.

show abstract

“…Although many works have targeted CLD in different access technologies like IEEE 802.15 [104], [105], IEEE 802.11 [106], LoRa [107] and cellular networks [108] and also employed AI & ML [61], [62], [63], there is no universal cross-layer design addressing all technologies. Some researchers have employed Multi-agent DRL (MARL) for intelligent RAT selection and resource allocation at the edge however, they did not consider cross-layer information from multiple layers for maximizing user QoS satisfactions [109], [110]. Due to significant differences in the MAC and PHY layers operations of RAT, getting crosslayer information and employing AI algorithms become increasingly complex.…”

Section: Role Of Ai and ML In Cross-layer Design In Multi-rat Networkmentioning

confidence: 99%

AI-Enabled Reliable QoS in Multi-RAT Wireless IoT Networks: Prospects, Challenges, and Future Directions

Zia

Chiumento

Havinga

2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Wireless IoT networks have seen an unprecedented rise in number of devices, heterogeneity and emerging use cases which led to diverse throughput, reliability and latency (Quality of Service) requirements. Fulfilling these diverse requirements in a rapidly changing and dynamic wireless environment is a complex and challenging task. On top of including new technologies and wireless standards, one solution is to deploy cross-layer Design (CLD) and multiple Radio Access Technologies (Multi-RAT) to develop scalable QoS-aware IoT networks. However, the complexity of such solutions is high as it involves complex inter-layer interactions and dependencies and inter-application dependencies in multi-RAT networks. Moreover, the wireless environment is very dynamic, so having an optimal constellation of parameters is a challenging task. In this paper, we address the possibilities of using Artificial Intelligence (AI) and Machine Learning (ML) to address these high dimensional and dynamic problems. Based on our findings, we have proposed a distributed network management framework employing AI & ML for studying inter-layer dependencies and developing cross-layer design, traffic classification and traffic prediction at the edge devices for reliable QoS in multi-RAT IoT networks. A thorough discussion on future directions and emerging challenges related to our proposed framework has also been provided for further research in this field.INDEX TERMS QoS in IoT networks, AI & ML for cross-layer design, cross-layer optimization, reliable QoS, multi-RAT networks, edge intelligence.

show abstract

Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous Multi-RAT Networks

Cited by 23 publications

References 49 publications

Age of Sensing Empowered Holographic ISAC Framework for NextG Wireless Networks: A VAE and DRL Approach

Age of Sensing Empowered Holographic ISAC Framework for NextG Wireless Networks: A VAE and DRL Approach

Switching of Interfaces and Self-optimization of weights using Backpropagation ANN in WHN Enviornment

AI-Enabled Reliable QoS in Multi-RAT Wireless IoT Networks: Prospects, Challenges, and Future Directions

Contact Info

Product

Resources

About