A Countermeasure Against Random Pulse Jamming in Time Domain Based on Reinforcement Learning

Abstract

Pulse jamming is one of the common malicious jamming patterns that can significantly reduce the of wireless communication's reliability. This paper investigates the problem of anti-jamming communication in a random pulse jamming environment. In order to obtain the countermeasure in time domain, the Markov decision process (MDP) is employed to model and analyze the above problem, and a time-domain anti-pulse jamming algorithm (TDAA) based on reinforcement learning is proposed. The proposed algorithm learns from the dynamic interaction with the jamming environment to gradually approximate the optimal time-domain strategy. The optimal strategy enables the transmitter to switch between two states, i.e. ''active'' and ''silent'', to avoid random pulse jamming. In addition, a state estimation and adjustment method for the random pulse jamming environment is introduced to improve the robustness of the proposed TDAA. Simulation results show that, compared with continuous transmission, the proposed TDAA can effectively reduce the jamming collision ratio and significantly improve the normalized throughput. And compared with transmitting terminal Q-learning algorithm (TTQA), the proposed TDAA has higher time utilization ratio and normalized throughput.INDEX TERMS Anti-jamming, Markov decision process, Q-learning, reinforcement learning, random pulse jamming.

Anti-Jamming Game to Combat Intelligent Jamming for Cognitive Radio Networks

“…2. Different types of jammers discussed in the literature [28], namely, random jammer [29], constant jammer [20], reactive jammer [30], [31], sweep jammer [32], [33] and intelligent jammer [8] [34]. In recent years, game theory has been extensively used to model the dynamic interaction between legitimate users and the jammer.…”

“…Markov game is formulated to model and analyze the antijamming problem in multi user environment. A time domain countermeasures against random pulse jamming using MDP and reinforcement learning was presented in [29]. In [?…”

A Coalition Formation Game-Based Multi-User Grouping Approach in the Jamming Environment

“…The communication anti-jamming process is actually a game process between the wireless communication system and the jammer; therefore, the game theory used in antijamming technologies has attracted wide research interest [2][3][4]. In the existing literature, methods such as Stackelberg game [5], stochastic learning theory [6], and reinforcement learning [7][8][9] have been studied, from the aspects of frequency and power By building a game model between communication and jamming, and by implementing a continuous trial-and-error method to avoid jamming, the optimal communication anti-jamming strategy was determined. In ref.…”

A Distributed Anti-Jamming Algorithm Based on Actor–Critic Countering Intelligent Malicious Jamming for WSN

“…What is more, reinforcement learning, a vital branch of machine learning, is capable of ensuring anti-jamming communications based on joint actions executed by the WSN and the feedback from the external environment, without the need for jamming modeling [ 8 ]. Anti-jamming communication schemes based on reinforcement learning have already been extensively studied [ 2 , 7 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. In reference [ 7 ], a SARSA-based anti-jamming algorithm was proposed to counter random pulse jamming in a time domain; however, most of the current research focuses on interference patterns with frequency domain characteristics.…”

“…In reference [ 7 ], a SARSA-based anti-jamming algorithm was proposed to counter random pulse jamming in a time domain; however, most of the current research focuses on interference patterns with frequency domain characteristics. In references [ 9 , 10 ], an anti-jamming scheme based on Q-learning is presented, which is widely regarded as a classic scheme without the need of jamming modeling. However, it only considered the single-agent network scenario.…”