A Reinforcement Learning Approach for Dynamic Spectrum Anti-jamming in Fading Environment

Kong, Lijun; Xu, Yuhua; Zhang, Yuli; Pei, Xufang; Ke, Mingxing; Wang, Ximing; Bai, Wei; Feng, Zhibin

doi:10.1109/icct.2018.8600218

Cited by 26 publications

(37 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compare this paper to our previous works [27,28], which studied anti-jamming channel selection in wireless communication networks, and to our previous works [29], the main differences are: i) Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 13 September 2018 doi:10.20944/preprints201809.0227.v1 Work [27] investigated the multi-agent learning method for anti-jamming problem, and work [28] considered the single reinforcement learning in fading environment. However, both these two works did not take the mobility of UAVs into consideration.…”

Section: Introductionsupporting

confidence: 58%

Location-aware Cooperative Anti-jamming Distributed Channel Selection Approach in UAV Communication Networks

Xu¹,

Ren²,

Jin³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

This paper investigates the cooperative anti-jamming distributed channel selection problem in UAV communication networks. Considering the existence of malicious jamming and co-channel interference, a location-aware cooperative anti-jamming scheme is designed for the purpose of maximizing the users' utilities. Users in the UAV group cooperate with each other via location information sharing. When the received interference energy is lower than mutual interference threshold, users conduct channel selection strategies independently. Otherwise, users take joint actions with a cooperative anti-jamming pattern under the impact of mutual interference. Aimed at the independent anti-jamming channel selection problem under no mutual interference, a Markov Decision Process framework is introduced, whereas for the cooperative anti-jamming channel selection case under the influence of co-channel mutual interference, a Markov game framework is employed. Furthermore, motivated by reinforcement learning with a ``Cooperation-Decision-Feedback-Adjustment" idea, we design a location-aware cooperative anti-jamming distributed channel selection algorithm (LCADCSA) to obtain the optimal anti-jamming channel strategies for the users with a distributed way. In addition, the channel switching cost and cooperation cost, which have great impact on the users' utilities, are introduced. Finally, simulation results show that the proposed algorithm converges to a stable solution with which the UAV group can avoid the malicious jamming as well as co-channel interference effectively.

show abstract

Section: Introductionsupporting

confidence: 58%

Location-aware Cooperative Anti-jamming Distributed Channel Selection Approach in UAV Communication Networks

Xu¹,

Ren²,

Jin³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As is shown in Equation 2, Tr n a n (t), a −n (t), a j (t) depicts the user n's throughput under the threat of malicious jamming and co-channel interference, and in Equation 3, the congestion degree I n (c n , t) reflects the number of users who are influenced by co-channel interference. Different from [25,31], this work consider the channel switching of users and introduces the channel switching cost unit W s to evaluate the performance loss. Moreover, if users cooperate with each other and take actions jointly, a cooperation cost unit W c is also brought in.…”

Section: System Model and Problem Formulationmentioning

confidence: 99%

“…Motivated by [22,24,25], we think the single Q-learning method is suitable for the case where the UAV group is not influenced by co-channel mutual interference. In the traditional single-Q learning algorithm, every user maintains and updates its independent Q table Q n ; for user n, the updating process of Q function is shown as:…”

Section: Single Q-learningmentioning

confidence: 99%

“…However, this work did not take the co-channel interference into consideration. In [24], the author dealt with the channel selection problem for cognitive networks via cooperative Q-learning, and in [25], a dynamic spectrum anti-jamming method in a fading environment was investigated. Furthermore, in [26], a deep Q network was built, and the anti-jamming channel selection problem was solved using a deep Q-learning method.…”

Section: Introductionmentioning

confidence: 99%

“…Comparing this paper to our previous works [25,31], which studied the anti-jamming channel selection in wireless communication networks, and to our previous work [32], we summarize the main differences as: (i) The work in [31] investigated the multi-agent learning method for anti-jamming problem, and [25] considered the single reinforcement learning in a fading environment. However, neither of them took the mobility of UAVs into consideration; whereas in our paper, the anti-jamming channel selection approach in UAV communication networks is investigated, taking the mobility of UAVs into consideration, which causes the variation of the co-channel interference level.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Interference-Aware Cooperative Anti-Jamming Distributed Channel Selection in UAV Communication Networks

Ren

Chen

et al. 2018

Applied Sciences

Self Cite

View full text Add to dashboard Cite

This paper investigates the cooperative anti-jamming distributed channel selection problem in UAV communication networks. Considering the existence of malicious jamming and co-channel interference, we design an interference-aware cooperative anti-jamming scheme for the purpose of maximizing users’ utilities. Moreover, the channel switching cost and cooperation cost are introduced, which have a great impact on users’ utilities. Users in the UAV group sense the co-channel interference signal energy to judge whether they are influenced by co-channel interference. When the received co-channel interference signal energy is lower than the co-channel interference threshold, users conduct channel selection strategies independently. Otherwise, users cooperate with each other and take joint actions with a cooperative anti-jamming pattern under the impact of co-channel interference. Aiming at the independent anti-jamming channel selection problem under no co-channel interference, a Markov decision process framework is introduced, whereas for the cooperative anti-jamming channel selection case under the influence of co-channel mutual interference, a Markov game framework is employed. Furthermore, motivated by Q-learning with a “cooperation-decision-feedback-adjustment” idea, we design an interference-aware cooperative anti-jamming distributed channel selection algorithm (ICADCSA) to obtain the optimal anti-jamming channel strategies for users in a distributed way. In addition, a discussion on the quick decision for UAVs is conducted. Finally, simulation results show that the proposed algorithm converges to a stable solution with which the UAV group can avoid malicious jamming, as well as co-channel interference effectively and can realize a quick decision in high mobility UAV communication networks.

show abstract

A Q-Learning-Based Channel Selection and Data Scheduling Approach for High-Frequency Communications in Jamming Environment

Guo

et al. 2019

Machine Learning and Intelligent Communications

Self Cite

View full text Add to dashboard Cite

The existence of jammer and the limited buffer space bring major challenge to data transmission efficiency in high-frequency (HF) commuication. The data transmission problem of how to select transmission strategy with multi-channel and different buffer states to maximize the system throughput is studied in this paper. We model the data transmission problem as a Makov decision process (MDP). Then, a modified Q-learning with additional value is proposed to help transmitter to learn the appropriate strategy and improve the system throughput. The simulation results show the proposed Q-learning algorithm can converge to the optimal Q value. Simultaneously, the QL algorithm compared with the sensing algorithm has better system throughput and less packet loss.

show abstract

A Reinforcement Learning Approach for Dynamic Spectrum Anti-jamming in Fading Environment

Cited by 26 publications

References 22 publications

Location-aware Cooperative Anti-jamming Distributed Channel Selection Approach in UAV Communication Networks

Location-aware Cooperative Anti-jamming Distributed Channel Selection Approach in UAV Communication Networks

Interference-Aware Cooperative Anti-Jamming Distributed Channel Selection in UAV Communication Networks

A Q-Learning-Based Channel Selection and Data Scheduling Approach for High-Frequency Communications in Jamming Environment

Contact Info

Product

Resources

About