“One Plus One is Greater Than Two”: Defeating Intelligent Dynamic Jamming with Collaborative Multi-agent Reinforcement Learning

A Distributed Anti-Jamming Algorithm Based on Actor–Critic Countering Intelligent Malicious Jamming for WSN

“…In order to evaluate the performance of the DAJA algorithm applied for anti-intelligent jamming communication in wireless sensor networks, we adopt the method of control variable in this simulation. Except for the newly proposed structure of the timeslot and the parameters in the A–C algorithm, other parameters, e.g., the number of available channels , sensors, namely the agents in the system etc., are consistent with those in the previous work [ 12 ] of our team.…”

“…A single jamming lasts timeslots. However, compared with intelligent jamming assumptions used in reference [ 12 ], the jamming model used in this paper can change their jamming parameter periodically, which is more intelligent and can be more threatening to the communications among the sensor nodes. In this paper, we define this jamming pattern as multi-channel probabilistic tracking jamming (MPT-jamming).…”

“…The definition of the five elements in the MAMDP is shown as follows: is the number of sensors in this system, is the state space, is the action space, is the state transition probability, and is the immediate reward that the transmitters can receive from the system. The relevant formula in this section is based on reference [ 12 ].…”

“…What is more, reinforcement learning, a vital branch of machine learning, is capable of ensuring anti-jamming communications based on joint actions executed by the WSN and the feedback from the external environment, without the need for jamming modeling [ 8 ]. Anti-jamming communication schemes based on reinforcement learning have already been extensively studied [ 2 , 7 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. In reference [ 7 ], a SARSA-based anti-jamming algorithm was proposed to counter random pulse jamming in a time domain; however, most of the current research focuses on interference patterns with frequency domain characteristics.…”

“…However, in these studies, the jamming modeling was still based on the traditional sweep jamming, which shows a lot of regularity and can be easily learned and mastered by a WSN. The jamming model used in reference [ 12 ] features pseudorandom characteristics, which can be regarded as a kind of intelligent jamming. In reference [ 13 ], a Dyna-Q-based algorithm was proposed to obtain better adaptability and faster convergence speed by updating the value function and adjusting the external environment model in real time, while the external environment model is fixed in the classical Q-learning process.…”