Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

Zeng, Yi; Pan, Minzhou; Anh, Just, Hoang; Lyu, Lingjuan; Qiu, Min; Jia, Ruoxi

doi:10.48550/arxiv.2204.05255

Cited by 5 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Souri et al 2021) allowed hidden attacks on training-from-scratch models by applying gradient matching. Recently, Narcissus (Zeng et al 2022) employed a clean surrogate model and optimized an uniform trigger; that approach is quite similar to adversarial attack. Still, all these methods had underwhelming attack performance compared to dirty-label counterparts.…”

Section: Previous Backdoor Attacksmentioning

confidence: 99%

“…Next, we compare our method with the existing cleanlabel attacks on CIFAR-10 in Table 2. The baselines include BadNets (Gu, Dolan-Gavitt, and Garg 2017), Labelconsistent (Turner, Tsipras, and Madry 2019), SIG (Barni, Kallas, and Tondi 2019), Sleeper Agent (Souri et al 2021), and Narcissus (Zeng et al 2022). Note that Sleeper Agent Table 2: Comparison between clean-label attacks on CIFAR-10.…”

Section: Attack Experimentsmentioning

confidence: 99%

“…In addition, most existing clean-label attacks (Turner, Tsipras, and Madry 2019; Barni, Kallas, and Tondi 2019) cannot reach an 80% ASR. Although some recent works (Ning et al 2021;Zeng et al 2022) achieves near-perfect ASRs, they require significant modifications to the training data. Due to the difficulty in designing a working and effective algorithm, only a few cleanlabel attacks have been proposed, and they have not been well studied in backdoor defense research.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

COMBAT: Alternated Training for Effective Clean-Label Backdoor Attacks

Huynh,

Nguyen,

Pham

et al. 2024

AAAI

View full text Add to dashboard Cite

Backdoor attacks pose a critical concern to the practice of using third-party data for AI development. The data can be poisoned to make a trained model misbehave when a predefined trigger pattern appears, granting the attackers illegal benefits. While most proposed backdoor attacks are dirty-label, clean-label attacks are more desirable by keeping data labels unchanged to dodge human inspection. However, designing a working clean-label attack is a challenging task, and existing clean-label attacks show underwhelming performance. In this paper, we propose a novel mechanism to develop clean-label attacks with outstanding attack performance. The key component is a trigger pattern generator, which is trained together with a surrogate model in an alternating manner. Our proposed mechanism is flexible and customizable, allowing different backdoor trigger types and behaviors for either single or multiple target labels. Our backdoor attacks can reach near-perfect attack success rates and bypass all state-of-the-art backdoor defenses, as illustrated via comprehensive experiments on standard benchmark datasets. Our code is available at https://github.com/VinAIResearch/COMBAT.

show abstract

Section: Previous Backdoor Attacksmentioning

confidence: 99%

Section: Attack Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

COMBAT: Alternated Training for Effective Clean-Label Backdoor Attacks

Huynh,

Nguyen,

Pham

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…For example, [55] employed a backdoor generation network to generate an invisible backdoor pattern for specific input. On the other hand, many studies are focused on contaminating datasets by clean label data [56] [57] [58] to evade the manual dataset reviews.…”

Section: Related Workmentioning

confidence: 99%

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Zhu¹,

Tang²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Most existing methods to detect backdoored machine learning (ML) models take one of the two approaches: trigger inversion (aka. reverse engineer) and weight analysis (aka. model diagnosis). In particular, the gradientbased trigger inversion is considered to be among the most effective backdoor detection techniques, as evidenced by the TrojAI competition [1], Trojan Detection Challenge [2] and backdoorBench [3]. However, little has been done to understand why this technique works so well and, more importantly, whether it raises the bar to the backdoor attack. In this paper, we report the first attempt to answer this question by analyzing the change rate of the backdoored model around its trigger-carrying inputs. Our study shows that existing attacks tend to inject the backdoor characterized by a low change rate around trigger-carrying inputs, which are easy to capture by gradient-based trigger inversion. In the meantime, we found that the low change rate is not necessary for a backdoor attack to succeed: we design a new attack enhancement called Gradient Shaping (GRASP), which follows the opposite direction of adversarial training to reduce the change rate of a backdoored model with regard to the trigger, without undermining its backdoor effect. Also, we provide a theoretic analysis to explain the effectiveness of this new technique and the fundamental weakness of gradient-based trigger inversion. Finally, we perform both theoretical and experimental analysis, showing that the GRASP enhancement does not reduce the effectiveness of the stealthy attacks against the backdoor detection methods based on weight analysis, as well as other backdoor mitigation methods without using detection.

show abstract

“…However, in IoT applications, federated learning is vulnerable to malicious attacks [ 12 , 13 ], such as backdoor attacks [ 13 , 14 , 15 , 16 , 17 , 18 ]. Backdoor attacks during model training can skew the model to produce specific erroneous outputs on targeted inputs.…”

Section: Introductionmentioning

confidence: 99%

Edge-Cloud Collaborative Defense against Backdoor Attacks in Federated Learning

Yang

Zheng

Wang

et al. 2023

Sensors

View full text Add to dashboard Cite

Federated learning has a distributed collaborative training mode, widely used in IoT scenarios of edge computing intelligent services. However, federated learning is vulnerable to malicious attacks, mainly backdoor attacks. Once an edge node implements a backdoor attack, the embedded backdoor mode will rapidly expand to all relevant edge nodes, which poses a considerable challenge to security-sensitive edge computing intelligent services. In the traditional edge collaborative backdoor defense method, only the cloud server is trusted by default. However, edge computing intelligent services have limited bandwidth and unstable network connections, which make it impossible for edge devices to retrain their models or update the global model. Therefore, it is crucial to detect whether the data of edge nodes are polluted in time. This paper proposes a layered defense framework for edge-computing intelligent services. At the edge, we combine the gradient rising strategy and attention self-distillation mechanism to maximize the correlation between edge device data and edge object categories and train a clean model as much as possible. On the server side, we first implement a two-layer backdoor detection mechanism to eliminate backdoor updates and use the attention self-distillation mechanism to restore the model performance. Our results show that the two-stage defense mode is more suitable for the security protection of edge computing intelligent services. It can not only weaken the effectiveness of the backdoor at the edge end but also conduct this defense at the server end, making the model more secure. The precision of our model on the main task is almost the same as that of the clean model.

show abstract

Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

Cited by 5 publications

References 26 publications

COMBAT: Alternated Training for Effective Clean-Label Backdoor Attacks

COMBAT: Alternated Training for Effective Clean-Label Backdoor Attacks

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Edge-Cloud Collaborative Defense against Backdoor Attacks in Federated Learning

Contact Info

Product

Resources

About