Trojan Signatures in DNN Weights

Fields, Gregory; Samragh, Mohammad; Javaheripi, Mojan; Koushanfar, Farinaz; Javidi, Tara

doi:10.1109/iccvw54120.2021.00008

Cited by 16 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Weight analysis aims to distinguish backdoor and benign models by analyzing the signals in model parameters. Specifically, such distinguishable signals are first retrieved, often through training a classifier on the parameters of some sample models, which are subsequently utilized to predict whether any given model is backdoored [9][5] [8]. In this section, we evaluated the effectiveness of GRASP against the weight analysisbased backdoor detection methods, which have been shown to perform well in the recent backdoor competitions [14], [1].…”

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

“…Here we selected Trojan Signature (TS) [9], MNTD [47], Activation Clustering (AC) [5] , Beatrix [30] and ABS [28], the representative methods based on weight analysis. We computed their AUCs on 20 models using VGG16, including ten clean models and ten backdoored models, respectively, trained on each of the three datasets (CIFA-10, MNIST, and Tiny Im-ageNet).…”

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Zhu,

Tang,

Tang

et al. 2024

Proceedings 2024 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Zhu,

Tang,

Tang

et al. 2024

Proceedings 2024 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

“…of sample models, and then utilized to predict whether any given model is backdoored [48][8] [49]. In this section, we present theoretical and experimental studies to show that backdoored models poisoned by a GRASP-enhanced attack are not further away from the benign models of the same primary task than the backdoored model poisoned by the same attack without GRASP enhancement.…”

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

“…We then evaluated the effectiveness of GRASP against the weight analysis-based backdoor detection, which has been adopted by some teams and performed well in some cases in recent backdoor competitions [1], [2]. Here we selected Trojan Signature (TS) [48], MNTD [9], Activation Clustering (AC) [50] and ABS [6], the representative meth-ods based on weight analysis. We computed their AUCs on 20 models using VGG16, including ten clean models and ten backdoored models, respectively, trained on each of the three datasets (CIFA-10, MNIST, and GTSRB).…”

Section: Against Weight Analysis Detectionmentioning

confidence: 99%

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Zhu¹,

Tang²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Most existing methods to detect backdoored machine learning (ML) models take one of the two approaches: trigger inversion (aka. reverse engineer) and weight analysis (aka. model diagnosis). In particular, the gradientbased trigger inversion is considered to be among the most effective backdoor detection techniques, as evidenced by the TrojAI competition [1], Trojan Detection Challenge [2] and backdoorBench [3]. However, little has been done to understand why this technique works so well and, more importantly, whether it raises the bar to the backdoor attack. In this paper, we report the first attempt to answer this question by analyzing the change rate of the backdoored model around its trigger-carrying inputs. Our study shows that existing attacks tend to inject the backdoor characterized by a low change rate around trigger-carrying inputs, which are easy to capture by gradient-based trigger inversion. In the meantime, we found that the low change rate is not necessary for a backdoor attack to succeed: we design a new attack enhancement called Gradient Shaping (GRASP), which follows the opposite direction of adversarial training to reduce the change rate of a backdoored model with regard to the trigger, without undermining its backdoor effect. Also, we provide a theoretic analysis to explain the effectiveness of this new technique and the fundamental weakness of gradient-based trigger inversion. Finally, we perform both theoretical and experimental analysis, showing that the GRASP enhancement does not reduce the effectiveness of the stealthy attacks against the backdoor detection methods based on weight analysis, as well as other backdoor mitigation methods without using detection.

show abstract

“…Trojan Signatures [30] is another approach in white-box setting. Trojan Signature provides a highly effective, uniquely lightweight detection method and it needs no data, little computation, and applies to many different types of triggers.…”

Section: White-box Backdoor Detectionmentioning

confidence: 99%

An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks

Zhang¹,

Chen²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

With the surge of Machine Learning (ML), An emerging amount of intelligent applications have been developed. Deep Neural Networks (DNNs) have demonstrated unprecedented performance across various fields such as medical diagnosis and autonomous driving. While DNNs are widely employed in security-sensitive fields, they are identified to be vulnerable to Neural Trojan (NT) attacks that are controlled and activated by stealthy triggers. In this paper, we target to design a robust and adaptive Trojan detection scheme that inspects whether a pre-trained model has been Trojaned before its deployment. Prior works are oblivious of the intrinsic property of trigger distribution and try to reconstruct the trigger pattern using simple heuristics, i.e., stimulating the given model to incorrect outputs. As a result, their detection time and effectiveness are limited. We leverage the observation that the pixel trigger typically features spatial dependency and propose the first trigger approximation based black-box Trojan detection framework that enables fast and scalable search of the trigger in the input space. Furthermore, our approach can also detect Trojans embedded in the feature space where certain filter transformations are used to activate the Trojan. We perform extensive experiments to investigate the performance of our approach across various datasets and ML models. Empirical results show that our approach achieves a ROC-AUC score of 0.93 on the public TrojAI dataset . Our code can be found in https://github.com/xinqiaozhang/adatrojan

show abstract

Trojan Signatures in DNN Weights

Cited by 16 publications

References 7 publications

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering

An Adaptive Black-box Backdoor Detection Method for Deep Neural Networks

Contact Info

Product

Resources

About