Reinforcement Learning for Mixed Cooperative/Competitive Dynamic Spectrum Access

Bowyer, Caleb M.; Greene, David J.; Ward, Tyler; Menendez, Marco; Shea, John M.; Wong, T.F.

doi:10.1109/dyspan.2019.8935725

Cited by 17 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several papers have applied RL to the MAC layer, mostly to solve radio resource management (RRM) problems such as scheduling ( [8], [9]) and dynamic spectrum access ( [10], [11]).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning

Mota,

Valcarce,

Gorce

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a new framework, exploiting the multi-agent deep deterministic policy gradient (MAD-DPG) algorithm, to enable a base station (BS) and user equipment (UE) to come up with a medium access control (MAC) protocol in a multiple access scenario. In this framework, the BS and UEs are reinforcement learning (RL) agents that need to learn to cooperate in order to deliver data. The network nodes can exchange control messages to collaborate and deliver data across the network, but without any prior agreement on the meaning of the control messages. In such a framework, the agents have to learn not only the channel access policy, but also the signaling policy. The collaboration between agents is shown to be important, by comparing the proposed algorithm to ablated versions where either the communication between agents or the central critic is removed. The comparison with a contentionfree baseline shows that our framework achieves a superior performance in terms of goodput and can effectively be used to learn a new protocol.

show abstract

“…Several papers have applied RL to the MAC layer, mostly to solve radio resource management (RRM) problems such as scheduling ( [8], [9]) and dynamic spectrum access ( [10], [11]).…”

Section: Related Workmentioning

confidence: 99%

“…+𝜌, if a new SDU was received by the BS −𝜌, if a UE deleted a SDU that has not been received by the BS −1, else, (10) where 𝜌 is a positive integer. This choice of reward is possible by leveraging the CTDE.…”

Section: A Marl Formulationmentioning

confidence: 99%

The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning

Mota,

Valcarce,

Gorce

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Since these approaches emulate a centralized decision they can achieve good results. For example the First and third place winners of the recent DARPA spectrum collaboration challenge(teams GatorWings [14] and Zylinium [15]) used a leader election based control. GatorWings modeled the problem as a Partially Observable Markov Decision Process (POMDP) and implemented a SARSA algorithm using a Deep Neural Network (DNN).…”

Section: A Quasi-centralized Algorithmsmentioning

confidence: 99%

Medium Access Control protocol for Collaborative Spectrum Learning in Wireless Networks

Boyarski¹,

Leshem²

2021

Preprint

View full text Add to dashboard Cite

In recent years there is a growing effort to provide learning algorithms for spectrum collaboration. In this paper we present a medium access control protocol which alllows spectrum collaboration with minimal regret and high spectral efficiency in highly loaded networks. We present a fully-distributed algorithm for spectrum collaboration in congested ad-hoc networks. The algorithm jointly solves both the channel allocation and access scheduling problems. We prove that the algorithm has an optimal logarithmic regret. Based on the algorithm we provide a medium access control protocol which allows distributed implementation of the algorithm in ad-hoc networks. The protocol utilizes single-channel opportunistic carrier sensing to carry out a lowcomplexity distributed auction in time and frequency. We also discuss practical implementation issues such as bounded frame size and speed of convergence. Computer simulations comparing the algorithm to state-of-the-art distributed medium access control protocols show the significant advantage of the proposed scheme.

show abstract

“…In recent years, DRL has been used for Dynamic Spectrum Access (DSA) and high performance has recently been achieved by subdividing the task into the smaller sub-problems of channel selection, admission control, and scheduling [8].…”

Section: Related Workmentioning

confidence: 99%

Towards A Machine Learning-Based Framework For Automated Design of Networking Protocols

Pasandi

2019

2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)

View full text Add to dashboard Cite

Networking protocols are designed through long-time and hard-work human eorts. Machine Learning (ML)-based solutions have been developed for communication protocol design to avoid manual eorts to tune individual protocol parameters. While other proposed ML-based methods mainly focus on tuning individual protocol parameters (e.g., adjusting contention window), our main contribution is to propose a novel Deep Reinforcement Learning (DRL)-based framework to systematically design and evaluate networking protocols. We decouple a protocol into a set of parametric modules, each representing a main protocol functionality that is used as DRL input to better understand the generated protocols design optimization and analyze them in a systematic fashion. As a case study, we introduce and evaluate Deep-MAC a framework in which a MAC protocol is decoupled into a set of blocks across popular avors of 802.11 WLANs (e.g., 802.11 b/a/g/n/ac). We are interested to see what blocks are selected by DeepMAC across dierent networking scenarios and whether DeepMAC is able to adapt to network dynamics.

show abstract

Reinforcement Learning for Mixed Cooperative/Competitive Dynamic Spectrum Access

Cited by 17 publications

References 3 publications

The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning

The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning

Medium Access Control protocol for Collaborative Spectrum Learning in Wireless Networks

Towards A Machine Learning-Based Framework For Automated Design of Networking Protocols

Contact Info

Product

Resources

About