Toward Joint Learning of Optimal MAC Signaling and Wireless Channel Access

Abstract

Communication protocols are the languages used by network nodes to accomplish their tasks. Before a User Equipment (UE) can exchange a data bit with the Base Station (BS), it must first negotiate the conditions and parameters for that transmission, which is supported by signaling messages at all layers of the protocol stack. Each year, the mobile communications industry invests large sums of money to define and standardize these messages, which are designed by humans during lengthy technical (and often political) debates. But is this the only way to develop a protocol? Could machines emerge their own signaling protocols automatically and without human intervention? We address the question of whether radios can learn a pre-given target language (i.e., a signaling) as an intermediate step towards evolving their own. Furthermore, we train cellular radios to emerge a channel access policy that performs optimally under the constraints of the target signaling. We show that multi-agent reinforcement learning (MARL) and learning-to-communicate techniques achieve this goal with gains over expert systems. Finally, we provide insight into the transferability of these results to scenarios never seen during training.

Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

“…Machine learning techniques have been used to solve various problems in communications systems. In [2][3][4][5][6] some interesting use-cases of machine learning in the field of wireless communication and networking are surveyed: MAC layer protocols designed with reinforcement learning, deep neural networks for MIMO detection, UE positioning with neural networks, and others. In [6], the authors address the problem of designing signaling protocols for the MAC layer using reinforcement learning.…”

“…In [2][3][4][5][6] some interesting use-cases of machine learning in the field of wireless communication and networking are surveyed: MAC layer protocols designed with reinforcement learning, deep neural networks for MIMO detection, UE positioning with neural networks, and others. In [6], the authors address the problem of designing signaling protocols for the MAC layer using reinforcement learning. The results show promising future for nonhuman-made protocols, they are faster and cheaper to construct when compared to the ones standardized by humans.…”

Learning Generalized Wireless MAC Communication Protocols via Abstraction

“…Recent advances in deep RL and learning-to-communicate techniques (e.g., Differentiable Inter-Agent Learning (DIAL), and Reinforced Inter-Agent Learning (RIAL) [3]) have led to the emergence of protocol learning for the physical (PHY) and media access control (MAC) layers [4]- [7]. Among them, to the best of our knowledge, the only works that assess the problem of MAC protocol learning in both control and data planes are [4], [5]. Therein, user equipments (UEs) are cast as agents that learn from their partial observation of the global state how to deliver MAC protocol data units (PDUs) to the base station (BS) throughout the radio channel.…”

“…Specifically, in [4], both the BS and the UEs are cast as RL agents, and the multi-agent deep deterministic policy gradient (MADDPG) algorithm is adopted, that is, a commonly used CTDE-based actor-critic method. In [5], the BS is modeled as an expert agent adopting a predefined protocol, while the UEs are RL agents trained to learn a shared channel-access policy following the target signaling policy set by the BS. The policy is learned by exploiting a tabular Q-learning algorithm that follows the CTDE paradigm.…”

“…The policy is learned by exploiting a tabular Q-learning algorithm that follows the CTDE paradigm. However, despite the good performances showed in the training environments, the learned protocols (i.e., policies) fail to generalize outside of their training distribution, as showed in [5].…”