Distributed Multichannel Access in High-Frequency Diversity Networks: A Multi-Agent Learning Approach With Correlated Equilibrium

Abstract

This paper investigates the problem of multiuser multichannel access in distributed high-frequency (HF) diversity communication networks using a game-theoretic learning algorithm which is based on correlation equilibrium (CE). We formulate the channel access problem in the HF networks as a non-cooperative game. In the access game, each user equipment (UE) optimizes its access strategy without the information about other UEs, which makes the channel access problem challenging. It is proved that there is at least one CE point that makes all UEs' access strategy efficient and fair. We propose a distributed learning algorithm based on CE to achieve multiuser access with low cost and fairly in the distributed HF networks. We use coordination signals to help each UE learn the access strategy by themselves. When each UE receives and recognizes the right coordination signals, UEs will learn to transmit data on right channels without further collisions after the learning phase. The simulation results show that the proposed learning algorithm can not only completely avoid interference and get optimal throughput but also guarantee fairness among all UEs. INDEX TERMS HF communication networks, multichannel access, game theory, correlation equilibrium (CE). The associate editor coordinating the review of this manuscript and approving it for publication was Wenchi Cheng. which is affected by many factors, like sunspot, season and region, the point-to-point communication is really unreliable. The HF communication networks which connect the HF infrastructures through the wired network, e.g., the IP networks, were proposed to enhance communication performance [10]-[12]. At the same time, diversity techniques are widely deployed in HF communications, among which channel diversity is a promising way by transmitting the same signal in several different channels [12]-[14]. However, all user equipments (UEs) have to access multichannel which increases the collision probability and the available HF channels are scant. Thus, channel diversity brings a new problem that how to optimize user's access strategy to improve network communication performance. In this article, the multichannel access problem with multiple UEs is investigated in distributed HF diversity networks.

Achieving Correlated Equilibrium by Studying Opponent’s Behavior Through Policy-Based Deep Reinforcement Learning

“…The author in [24] proposed a distributed cooperation policy selection scheme for interference to perform subcarrier assignment for uplink multi-cell OFDMA systems by adopting the correlated equilibrium solution concept that achieves better performance by allowing each user to consider the joint distribution among users' actions to minimize the interference. Also, in [16], the author showed by using a game-theoretic learning algorithm which is based on correlation equilibrium in the problem of multi-user multichannel access in distributed high-frequency diversity communication networks can completely avoid interference and get optimal throughput. Moreover, it also guarantees fairness among all user equipment.…”

“…Although few articles tried to apply machine learning with correlated equilibrium solution concept in some applications, there are concerns within their research when applying to real-life scenarios. For example, some research showed that the Q-Learning method [14], [15] could achieve the correlated equilibrium within the system [16]- [18]. However, the issues with the Q-Learning method are high space and power consumption.…”