Attention-Based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices

Abstract

Due to great success of transformers in many applications such as natural language processing and computer vision, transformers have been successfully applied in automatic modulation classification. We have shown that transformer-based radio signal classification is vulnerable to imperceptible and carefully crafted attacks called adversarial examples. Therefore, we propose a defense system against adversarial examples in transformer-based modulation classifications. Considering the need for computationally efficient architecture particularly for Internet of Things (IoT)-based applications or operation of devices in environment where power supply is limited, we propose a compact transformer for modulation classification. The advantages of robust training such as adversarial training in transformers may not be attainable in compact transformers. By demonstrating this, we propose a novel compact transformer that can enhance robustness in the presence of adversarial attacks. The new method is aimed at transferring the adversarial attention map from the robustly trained large transformer to a compact transformer. The proposed method outperforms the state-of-the-art techniques for the considered white-box scenarios including fast gradient method and projected gradient descent attacks. We have provided reasoning of the underlying working mechanisms and investigated the transferability of the adversarial examples between different architectures. The proposed method has the potential to protect the transformer from the transferability of adversarial examples.

Artificial intelligence-powered intelligent reflecting surface systems countering adversarial attacks in machine learning

“…In neural network architecture and defensive distillation technique (DDT), the input data received from the user devices is used to IRS prediction method. Defensive distillation training networks is covered using a defended model which has deep neural networks with large network and shallow neural networks with small neural network [24], [25]. The overall system design for the proposed AI-powered intelligent reflecting surface system is shown in Figure 3.…”