Learning-based Spectrum Sensing and Access in Cognitive Radios via Approximate POMDPs

Abstract

A novel LEarning-based Spectrum Sensing and Access (LESSA) framework is proposed, wherein a cognitive radio (CR) learns a time-frequency correlation model underlying spectrum occupancy of licensed users (LUs) in a radio ecosystem; concurrently, it devises an approximately optimal spectrum sensing and access policy under sensing constraints. A Baum-Welch algorithm is proposed to learn a parametric Markov transition model of LUs' spectrum occupancy based on noisy spectrum measurements. Spectrum sensing and access are cast as a Partially-Observable Markov Decision Process, approximately optimized via randomized point-based value iteration. Fragmentation, Hamming-distance state filters and Monte-Carlo methods are proposed to alleviate the inherent computational complexity, and a weighted reward metric to regulate the trade-off between CR's throughput and interference to the LUs. Numerical evaluations demonstrate that LESSA performs within 5% of a genie-aided upper bound with foreknowledge of LUs' spectrum occupancy, and outperforms state-of-the-art algorithms across the entire trade-off region: 71% over correlation-based clustering, 26% over Neyman-Pearson-based spectrum sensing, 6% over the Viterbi algorithm, and 9% over adaptive Deep Q-Network. LESSA is then extended to a distributed Multi-Agent setting (MA-LESSA), by proposing novel neighbor discovery and channel access rank allocation. MA-LESSA improves CRs' throughputs by 43% over cooperative TD-SARSA, 84% over cooperative greedy distributed learning, and 3× over non-cooperative learning via g-statistics and ACKs. Finally, MA-LESSA is implemented on the DARPA SC2 platform, manifesting superior performance over competitors in a real-world TDWR-UNII WLAN emulation; its implementation feasibility is further validated on an ad-hoc distributed wireless testbed of ESP32 radios, exhibiting 96% success probability.