Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

Naparstek, Oshri; Cohen, Kobi

doi:10.48550/arxiv.1704.02613

Cited by 6 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There has been little prior work exploring the use of DRL to solve MAC problems, given that DRL itself is a new research topic. The MAC scheme in [14] employs DRL in homogeneous wireless networks. Specifically, [14] considered a network in which N radio nodes dynamically access K orthogonal channels using the same DRL MAC protocol.…”

Section: A Related Workmentioning

confidence: 99%

“…The MAC scheme in [14] employs DRL in homogeneous wireless networks. Specifically, [14] considered a network in which N radio nodes dynamically access K orthogonal channels using the same DRL MAC protocol. By contrast, we are interested in heterogeneous networks in which the DRL nodes must learn to collaborate with nodes employing other MAC protocols.…”

Section: A Related Workmentioning

confidence: 99%

“…Objectives can often be expressed as a function of several components, wherein each component can be expressed as a Q function (e.g., (14)). In the more general setup, the update equation of Q function still has the same form (i.e., (15) has the same form as (3)).…”

Section: B Dlma Reformulationmentioning

confidence: 99%

“…Returning to our wireless setting, the first term in ( 14) is the sum of local utility functions of all legacy nodes. Since the big agent (indexed by L + 1) is actually a combination of the K DRL nodes, and q (L+1) (•) /K is the estimated accumulated reward of each DRL node, the second term in (14) is the sum of local utility functions of all the DRL nodes. We have two remarks: i) the q (i) (s t , a t ) in ( 15) is an estimate of the expected accumulated discounted reward of node i (as expressed in (14), rather than the exact throughput x (i) in (13); ii) we use q (i) (s t , a t ) to help the agent make decisions because the exact throughput x (i) is not known.…”

Section: B Dlma Reformulationmentioning

confidence: 99%

See 3 more Smart Citations

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

Wang

Liew

2019

IEEE J. Select. Areas Commun.

298

181

View full text Add to dashboard Cite

This paper investigates a futuristic spectrum sharing paradigm for heterogeneous wireless networks with imperfect channels. In the heterogeneous networks, multiple wireless networks adopt different medium access control (MAC) protocols to share a common wireless spectrum and each network is unaware of the MACs of others. This paper aims to design a distributed deep reinforcement learning (DRL) based MAC protocol for a particular network, and the objective of this network is to achieve a global α-fairness objective. In the conventional DRL framework, feedback/reward given to the agent is always correctly received, so that the agent can optimize its strategy based on the received reward. In our wireless application where the channels are noisy, the feedback/reward (i.e., the ACK packet) may be lost due to channel noise and interference. Without correct feedback, the agent (i.e., the network user) may fail to find a good solution. Moreover, in the distributed protocol, each agent makes decisions on its own. It is a challenge to guarantee that the multiple agents will make coherent decisions and work together to achieve the same objective, particularly in the face of imperfect feedback channels. To tackle the challenge, we put forth (i) a feedback recovery mechanism to recover missing feedback information, and (ii) a two-stage action selection mechanism to aid coherent decision making to reduce transmission collisions among the agents. Extensive simulation results demonstrate the effectiveness of these two mechanisms. Last but not least, we believe that the feedback recovery mechanism and the two-stage action selection mechanism can also be used in general distributed multi-agent reinforcement learning problems in which feedback information on rewards can be corrupted.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: B Dlma Reformulationmentioning

confidence: 99%

Section: B Dlma Reformulationmentioning

confidence: 99%

See 2 more Smart Citations

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

Wang

Liew

2019

IEEE J. Select. Areas Commun.

298

181

View full text Add to dashboard Cite

show abstract

“…considered both the networking cost as well as the users' mean opinion score. In [277] and [278], He In order to curb the potentially excessive computational complexity resulting from having a large state space and to deal with its partial observability in cognitive radio networks, Naparstek and Cohen developed a distributed dynamic spectrum access scheme relying on deep multi-user reinforcement leaning, where each user maps his/her current state to spectrum access actions with the aid of a DQN for the sake of maximizing the network's utility which was achieved without any message exchanges [280]. Additionally, Wang et al [281] proposed an adaptive DQN algorithm for dynamic multichannel access, which was capable of achieving a near-optimal performance in complex scenarios.…”

Section: Deep Learning In Ngwnmentioning

confidence: 99%

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Wang,

Jiang,

Zhang

et al. 2019

Preprint

View full text Add to dashboard Cite

Next-generation wireless networks (NGWN) have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making.Hence, in this article, we review the thirty-year history of machine learning by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning, respectively. Furthermore, we investigate their employment in the compelling applications of NGWNs, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various machine learning algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.

show abstract

An efficient Deep reinforcement learning with extended Kalman filter for device‐to‐device communication underlaying cellular network

Khuntia

Hazra

2019

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In this paper, a novel Deep Q‐learning in addition with an extended Kalman filter (EKF) is proposed to solve the channel and power allocation issue for a device‐to‐device enabled cellular network, when the prior traffic information is not known to the base station. Furthermore, this paper work explores an optimal policy for resource and power allocation between users with the aim of maximizing the sum‐rate of the overall system. The proposed work comprises of four phases, ie, cell splitting, clustering, queuing model, channel allocation, and power allocation, simultaneously. The implementation of cell splitting with novel K‐means++ clustering technique increases the network coverage, reduces co‐channel cell interference, and minimizes the transmission power of nodes, whereas the M/M/C:N queuing model solves the issue of waiting time for users in a priority based data transmission. The difficulty with the Q‐learning and Deep Q‐learning environment is to achieve an optimal policy. This is because of the uncertainty of various parameters associated with the system, especially when the state space is extremely big. In order to improve the robustness of the learner, EKF together with the Deep Q‐network is presented in this paper, which incorporates weight uncertainty of the Q‐function as well as the state uncertainty during the transition. Furthermore, the use of EKF provides an improved version of the loss function that helps the learner in achieving an optimal policy. Through numerical simulation, the advantage of our resource sharing scheme over the other existing schemes is also verified.

show abstract

Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

Cited by 6 publications

References 0 publications

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

An efficient Deep reinforcement learning with extended Kalman filter for device‐to‐device communication underlaying cellular network

Contact Info

Product

Resources

About