Reinforcement Learning for Joint Channel/Subframe Selection of LTE in the Unlicensed Spectrum

Abstract

In recent years, to cope with the rapid growth in mobile data traffic, increasing the capacity of cellular networks is receiving more and more attention. To this end, offloading the current LTE-advanced or 5G system’s data traffic from licensed spectrum to the unlicensed spectrum that is used by WiFi systems, i.e., LTE-Licensed-Assisted-Access (LTE-LAA), has been extensively investigated. In the current LTE-LAA system, a Listen-Before-Talk (LBT) approach is implemented, which requires the LTE user also perform carrier sense before the transmission. However, fair LTE-WiFi coexistence is still hard to guarantee due to their unbalanced frame sizes and traffic loads. In the LTE-LAA system, the optimal channel selection and subframe number adjustment are the keys to realize efficient spectrum utilization and fair system coexistence. To this end, in this paper, we propose a reinforcement learning-based joint channel/subframe selection scheme for LTE-LAA. The proposed approach is implemented at the LTE access points with zero knowledge of the WiFi systems. The results of extensive simulations verify that the proposed approach can significantly improve the fairness and packet loss rate compared with baseline schemes.

Wi-Fi Meets ML: A Survey on Improving IEEE 802.11 Performance With Machine Learning

“…• adjust the TXOP duration of coexisting Wi-Fi and LTE-LAA systems based on buffered downlink data in APs and evolved Node Bs (eNBs) [216], and • select optimal channel and subframe numbers [236].…”

“…This approach not only improves performance in terms of LTE's rates but also in terms of reducing disturbances in Wi-Fi's performance and achieving coexistence fairness with Wi-Fi networks and other LTE-LAA operators. Finally, Kishimoto et al [236], use Q-learning for joint channel/subframe selection. Only LTE-LAA BSs perform learning and start with zero knowledge of neighboring Wi-Fi systems.…”

Wi-Fi Meets ML: A Survey on Improving IEEE 802.11 Performance with Machine Learning

“…Centralized collection of data regarding LTE-LAA and WiFi systems by the LTE cloud wireless access network (C-RAN) is proposed to support MMEs. Other papers implement distributed Q-learning to: (i) optimize spectral efficiency of WiFi/LTE-LAA coexistence [196], (ii) scale CW parameters depending on the collision probability observed in each backoff stage by LTE user entitys (UEs), as opposed to the legacy hybrid automatic repeat request (HARQ) mechanism implemented in cellular networks [200], (iii) select optimal TXOP and muting periods (i.e., giving opportunities for WiFi transmissions) which outperform random and round-robin mechanisms [190], (iv) adjust the TXOP duration of coexisting WiFi and LTE-LAA systems based on buffered downlink data in APs and evolved Node Bs (eNBs) [184], and (v) select optimal channel and subframe numbers [204]. In [184], both WiFi and LTE-LAA nodes serve as agents, which take actions (select TXOPs from 4, 6, 8 and 10 ms) and calculate rewards based on the target occupancy ratio.…”

“…It is shown that this approach not only improves performance in terms of LTE rate but also in terms of reducing disturbances in WiFi's performance and achieving coexistence fairness with WiFi networks and other LTE-LAA operators. Finally, in [204], Q-learning is used for joint channel/sub-frame selection. In this work, only LTE-LAA BSs perform learning with zero knowledge of concurrent WiFi systems.…”