Anti-Jamming Communication Using Slotted Cross Q Learning

Abstract

Most of the existing intelligent anti-jamming communication algorithms model sensing, learning, and transmission as a serial process, and ideally assume that the duration of sensing and learning timeslots is very short, almost negligible. However, when the jamming environment changes rapidly, the sensing and learning time can no longer be ignored, and the adaptability of the wireless communication system to the time-varying jamming environment will be significantly reduced. To solve this problem, this paper proposes a parallel Q-learning (PQL) algorithm. In the case of long sensing and learning time, by modeling sensing, learning, and transmission as parallel processes, the time that the transmitter remains silent during sensing and learning is reduced. Aiming at the situation that the PQL algorithm is susceptible to jamming when the jamming changes faster, this paper proposes an intelligent anti-jamming algorithm for wireless communication based on Slot Cross Q-learning (SCQL). In the case of rapid change of jamming channel, the system can sense and learn the jamming patterns in multiple successive jamming periods at the same time in the same timeslot, and use multiple Q-tables to learn the jamming patterns in different jamming periods, so as to achieve the effect of reliable communication in the environment with rapid change of jamming. The simulation results show that the jamming collision rate of the proposed algorithm under the condition of intelligent blocking jamming is equivalent to that of the traditional Q-learning (QL), but the timeslot utilization rate is higher. Compared with PQL, the proposed algorithm has the same slot utilization and lower jamming collision rate. Compared with random frequency hopping (RFH) anti-jamming, the proposed algorithm not only has higher timeslot utilization, but also has lower jamming collision rate.

Learning-Based Multi-Domain Anti-Jamming Communication with Unknown Information

“…Antijamming techniques in the power domain are regarded as the most direct and effective anti-jamming schemes and have been widely used [8]. In recent work, an intelligent anti-jamming algorithm based on Slot Cross Q-Learning (SCQL) was proposed in [17] to deal with the time-varying jamming environment where the jamming channel changed rapidly. In addition to the anti-jamming techniques in a certain single domain, in [5], an anti-jamming scheme in the spectrum-power domain has been proposed, which formulated a power control game and utilized a multi-armed bandit-based method to select the communication channel.…”

Anti-Jamming Communication Using Imitation Learning

“…In our previous work, we mainly studied the intelligent anti-jamming communication technology based on RL [5,24] and DQN [25], improved some technical details, and combined these with UCB technology. However, in this process, we find that both RL and DQN are faced with the dilemma of exploration and utilization, which requires a lot of time to explore the environment, and the ability to explore the environment in the utilization stage is poor.…”