Power Allocation in Cellular Network Without Global CSI: Bayesian Reinforcement Learning Approach

Khoshkbari, Hesam; Pourahmadi, Vahid; Sheikhzadeh, Hamid

doi:10.1109/icee50131.2020.9260583

Cited by 4 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dynamic probabilistic networks, with their unique uncertainty knowledge representation and rich probabilistic expressive ability, have significant features in describing nonlinearity, temporality, evolution, and uncertainty, and have been gradually applied to the fields of classification, decision-making and prediction, information recovery, and expert systems [10]. The time series in a dynamic probabilistic network is discrete and is represented by a time slice, on each time slice is a probabilistic network, and there are associative dependencies between neighboring time slices with different nodes.…”

Section: Based On Dynamic Probabilistic Network To Achieve "Comprehen...mentioning

confidence: 99%

“Comprehensive Integration” Method Based on the Bayesian Network and the Big Data Using in the Construction of New Energy System

Yunfei Chu, Daye Wang

2024

jes

View full text Add to dashboard Cite

With the continuous application of the big data technology, “Comprehensive Integration” theory is an approach to management based on Rooted theory, which has been successfully applied to the management of large enterprises and research organizations of all kinds. State Grid Corporation of China in the management process of the construction of a new energy system, with a strong innovative, large-scale investment, many participating units, urgent needs and other characteristics of its management capabilities put forward higher requirements. In this paper, a system integration principle based on the Bayesian probabilistic approach and the big data is proposed to realize the engineering application of the theory of “Comprehensive Integration”. By splitting the management system of the new energy system construction, the unit composition of the system can be interpreted to form a complete system and node interaction Bayesian network structure. The algorithm proposed in this paper is implemented in Netica, a commercial statistical analysis software, and its effectiveness is verified, which provides an important theoretical basis for the engineering practice of the theory of “Comprehensive Integration”. Finally, the effectiveness of the Dynamic probabilistic network method was quantitatively verified in a lithium battery pack energy storage management example.

show abstract

Section: Based On Dynamic Probabilistic Network To Achieve "Comprehen...mentioning

confidence: 99%

“Comprehensive Integration” Method Based on the Bayesian Network and the Big Data Using in the Construction of New Energy System

Yunfei Chu, Daye Wang

2024

jes

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

“…In our previous work [22], we utilized Deep Q-learning (DQL) to perform user association between a TBS and a HAPS. Though we showed promising results, it is noteworthy that the global channel state information (CSI) was used in [22]. Moreover, the agent's performance under imperfect CSI was not satisfactory compared to its performance under perfect CSI.…”

Section: Related Workmentioning

confidence: 99%

“…where ŝ is the next state and max aPA Q pŝ, a | θq is the estimate of the next Q-function. The DQL method is an off-policy DRL method that updates its policy based on the maximum Q-value of the next state (22), regardless of the action taken in the environment. In other words, the behaviour policy (the policy used for selecting actions) and the target policy (the policy used for learning) are different.…”

Section: A Backgroundmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Sharifi,

Khoshkbari,

Kaddoum

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

In this paper, we devise a deep SARSA-learning (DSRL) user scheduling algorithm for a base station (BS) that uses a high-altitude platform station (HAPS) as a backup to serve multiple users in a wireless cellular network. Considering a realistic scenario, we assume that only the outdated channel state information (CSI) of the terrestrial base station (TBS) is available in our defined user scheduling problem. To do so, we define a user scheduling problem using a Markov decision process (MDP) framework, where the objective function is to maximize the sum-rate with a minimum number of active antennas at the HAPS. Our performance analysis shows that the sum-rate obtained with our proposed DSRL algorithm is close to the optimal sum-rate achieved with an exhaustive search method. We also develop a heuristic optimization method to solve the user scheduling problem at the BS. We show that for a scenario where perfect CSI is not available, our proposed DSRL algorithm outperforms the heuristic optimization method.

show abstract

“…In allocating wireless network resources, Khoshkbari et al (2020a) replaced the policy network of deep reinforcement learning with a Bayesian neural network. The vast and sparse action space is efficiently searched in link level throughput maximization Khoshkbari et al (2020b). used deep reinforcement learning based on Bayesian neural networks to allocate network energy according to the local channel state information (CSI) of users to enhance the overall rate of dense wireless networks.…”

mentioning

confidence: 99%

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning

Wang

Meng

Zhou

et al. 2022

Preprint

View full text Add to dashboard Cite

Reinforcement learning is a research paradigm that is commonly utilized to tackle problems involving sequential decision-making. Agents learn optimum policy from samples generated by interacting with uncertain environments. Bayesian reinforcement learning presents the uncertainty measure of probability significance based on sequential decision-making, which plays a proper auxiliary function in the optimization of agent control policies, attracting a lot of study attention. This paper provides an overview of contemporary Bayesian and reinforcement learning methods in this setting. We begin by outlining the many types of reinforcement learning and Bayesian approaches, as well as their characteristics. Then we review the most sophisticated Bayesian reinforcement learning approaches from the perspective of uncertainty measurement and control, as well as summarize and discuss various challenges and related research in its expansion to complicated control tasks. Applying the Bayesian technique with reinforcement learning in diverse disciplines are then discussed. Finally, we highlight the current challenges of Bayesian reinforcement learning and future research topics that might help promote Bayesian reinforcement learning.

show abstract

Power Allocation in Cellular Network Without Global CSI: Bayesian Reinforcement Learning Approach

Cited by 4 publications

References 5 publications

“Comprehensive Integration” Method Based on the Bayesian Network and the Big Data Using in the Construction of New Energy System

“Comprehensive Integration” Method Based on the Bayesian Network and the Big Data Using in the Construction of New Energy System

Deep Reinforcement Learning Approach for HAPS User Scheduling in Massive MIMO Communications

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning

Contact Info

Product

Resources

About