Bayesian Reinforcement Learning for Link-Level Throughput Maximization

Khoshkbari, Hesam; Pourahmadi, Vahid; Sheikhzadeh, Hamid

doi:10.1109/lcomm.2020.2990308

Cited by 6 publications

(4 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The obtained action assigns a subset of users to be served by the HAPS and the remaining ones are served by the TBS. We use (18) to obtain the reward value Rps t , a t q. Afterward, the time-varying channels evolve based on (28).…”

Section: B Our Proposed Dsrl Methodsmentioning

confidence: 99%

“…Further, the authors perform user association to maximize the total throughput of the network while avoiding frequent handoffs resulting from the mobility of airborne vehicles. The authors of [18] studied the DRL-based user association in a VHetNet with en emphasize on the role of satellites. In [19], the authors study minimizing the age of information (AoI) in an intelligent transportation system (ITS) where UAVs collect the produced information by sensors on the vehicles to provide up-to-date data.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Sharifi,

Khoshkbari,

Kaddoum

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

In this paper, we devise a deep SARSA-learning (DSRL) user scheduling algorithm for a base station (BS) that uses a high-altitude platform station (HAPS) as a backup to serve multiple users in a wireless cellular network. Considering a realistic scenario, we assume that only the outdated channel state information (CSI) of the terrestrial base station (TBS) is available in our defined user scheduling problem. To do so, we define a user scheduling problem using a Markov decision process (MDP) framework, where the objective function is to maximize the sum-rate with a minimum number of active antennas at the HAPS. Our performance analysis shows that the sum-rate obtained with our proposed DSRL algorithm is close to the optimal sum-rate achieved with an exhaustive search method. We also develop a heuristic optimization method to solve the user scheduling problem at the BS. We show that for a scenario where perfect CSI is not available, our proposed DSRL algorithm outperforms the heuristic optimization method.

show abstract

Section: B Our Proposed Dsrl Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Sharifi,

Khoshkbari,

Kaddoum

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

show abstract

“…In [7], the authors proposed an interference coordination method for an integrated HAPS-terrestrial network that considered traffic load distribution between HAPS and terrestrial network. In [8], the authors developed a user association scheme for an integrated HAPS-terrestrial network based on the DQL approach considering delayed channel state information (CSI). In [9], we developed an iterative algorithm to design the subcarrier and transmit power allocation to user equipment (UEs) to handle the interference in vHetNets.…”

Section: Introductionmentioning

confidence: 99%

Handling Interference in Integrated HAPS-Terrestrial Networks Through Radio Resource Management

Alidadi

Yadav

Abbasi

et al. 2022

IEEE Wireless Commun. Lett.

View full text Add to dashboard Cite

Located in the stratospheric layer of Earth's atmosphere, the high altitude platform station (HAPS) is a promising network infrastructure, which can bring significant advantages to sixth-generation (6G) and beyond wireless communications systems by forming vertical heterogeneous networks (vHetNets). However, if not dealt with properly, integrated networks suffer from a number of performance challenges compared to standalone networks. In harmonized integrated networks, where different tiers share the same frequency spectrum, interference is an important challenge to be addressed. This work focuses on an integrated HAPSterrestrial network, serving users in an overlapped urban geographic area, and formulates a fairness optimization problem, aiming to maximize the minimum spectral efficiency (SE) of the network. Due to the highly nonconvex nature of the formulated problem, we develop a fast converging iterative algorithm that designs the massive multiple-input multiple-output (mMIMO) beamforming weights and the user association scheme such that the propagated inter-and intra-tier interference is managed. Simulation results demonstrate the proposed algorithm's superiority over standalone terrestrial networks and baseline algorithms.

show abstract

“…While receiving a scalar reward signal, a reinforcement learning agent interacts with the environment dynamically and adjusts its policy based on the data collected to maximize the expected return. Researchers have applied this paradigm to various scenarios, including robots Kober et al (2013); Donepudi (2020), manmachine games Silver et al (2016), instructional systems Iglesias et al (2009), recommendation systems Zhao et al (2018); Afsar et al (2021), resource management Mao et al (2016); Wang and Pedram (2016); Liessner et al (2019); Khoshkbari et al (2020a), medical diagnosis Yu et al (2021) and intelligent transportation Tang et al (2019); Haydari and Yilmaz (2022). As a result, solving decision-making problems in a variety of areas drives the study and implementation of reinforcement learning theories.…”

Section: Introductionmentioning

confidence: 99%

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning

Wang

Meng

Zhou

et al. 2022

Preprint

View full text Add to dashboard Cite

Reinforcement learning is a research paradigm that is commonly utilized to tackle problems involving sequential decision-making. Agents learn optimum policy from samples generated by interacting with uncertain environments. Bayesian reinforcement learning presents the uncertainty measure of probability significance based on sequential decision-making, which plays a proper auxiliary function in the optimization of agent control policies, attracting a lot of study attention. This paper provides an overview of contemporary Bayesian and reinforcement learning methods in this setting. We begin by outlining the many types of reinforcement learning and Bayesian approaches, as well as their characteristics. Then we review the most sophisticated Bayesian reinforcement learning approaches from the perspective of uncertainty measurement and control, as well as summarize and discuss various challenges and related research in its expansion to complicated control tasks. Applying the Bayesian technique with reinforcement learning in diverse disciplines are then discussed. Finally, we highlight the current challenges of Bayesian reinforcement learning and future research topics that might help promote Bayesian reinforcement learning.

show abstract

Bayesian Reinforcement Learning for Link-Level Throughput Maximization

Cited by 6 publications

References 4 publications

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Handling Interference in Integrated HAPS-Terrestrial Networks Through Radio Resource Management

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning

Contact Info

Product

Resources

About