Deep Q-network Based Reinforcement Learning for Distributed Dynamic Spectrum Access

Yadav, Manish Anand; Li, Yuhui; Fang, Guangjin; Shen, Bin

doi:10.1109/ccai55564.2022.9807797

Cited by 5 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We call an SU's behavior static if it adheres to only one channel, and dynamic if it is free to switch channels. While the concept of this model is simple, the design of spectrum sensing strategy faces various challenges: the interaction among multiple secondary users [5]- [7], spectrum sensing policy in the Markovian channels [8], [9], the trade-off between the cost of sequential sensing (when permitted) and the expected reward [10],…”

Section: Introductionmentioning

confidence: 99%

Minimizing File Transfer Time in Opportunistic Spectrum Access Model

Hu,

Doshi,

Eun

2022

Preprint

View full text Add to dashboard Cite

We study the file transfer problem in opportunistic spectrum access (OSA) model, which has been widely studied in throughput-oriented applications for max-throughput strategies and in delay-related works that commonly assume identical channel rates and fixed file sizes. Our work explicitly considers minimizing the file transfer time for a given file in a set of heterogeneous-rate Bernoulli channels, showing that max-throughput policy doesn't minimize file transfer time in general. We formulate a mathematical framework for static extend to dynamic policies by mapping our file transfer problem to a stochastic shortest path problem. We analyze the performance of our proposed static and dynamic optimal policies over the max-throughput policy. We propose a mixed-integer programming formulation as an efficient alternative way to obtain the dynamic optimal policy and show a huge reduction in computation time. Then, we propose a heuristic policy that takes into account the performance-complexity tradeoff and consider the online implementation with unknown channel parameters. Furthermore, we present numerical simulations to support our analytical results and discuss the effect of switching delay on different policies. Finally, we extend the file transfer problem to Markovian channels and demonstrate the impact of the correlation of each channel.

show abstract