Cassandra: Detecting Trojaned Networks From Adversarial Perturbations

Zhang, Xiaoyu; Gupta, Rohit; Mian, Ajmal; Rahnavard, Nazanin; Shah, Mubarak

doi:10.1109/access.2021.3101289

Cited by 10 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While recent work has shown that feature-space UAPs can be employed in attacks at training time, such as backdoor poisoning attacks [77], here we focus solely on the test phase of the machine learning pipeline. Specifically, we focus on evasion attacks ( §2.1) in which the attacker modifies objects at test-time in order to induce targeted misclassifications.…”

Section: Attack Scope and Objectivesmentioning

confidence: 99%

Realizable Universal Adversarial Perturbations for Malware

Labaca-Castro¹,

Muñoz-González²,

Pendlebury³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning classification models are vulnerable to adversarial examples-effective input-specific perturbations that can manipulate the model's output. Universal Adversarial Perturbations (UAPs), which identify noisy patterns that generalize across the input space, allow the attacker to greatly scale up the generation of these adversarial examples. Although UAPs have been explored in application domains beyond computer vision, little is known about their properties and implications in the specific context of realizable attacks, such as malware, where attackers must reason about satisfying challenging problem-space constraints.In this paper, we explore the challenges and strengths of UAPs in the context of malware classification. We generate sequences of problem-space transformations that induce UAPs in the corresponding feature-space embedding and evaluate their effectiveness across threat models that consider a varying degree of realistic attacker knowledge. Additionally, we propose adversarial training-based mitigations using knowledge derived from the problem-space transformations, and compare against alternative feature-space defenses. Our experiments limit the effectiveness of a white box Android evasion attack to ~20% at the cost of ~3% TPR at 1% FPR. We additionally show how our method can be adapted to more restrictive application domains such as Windows malware.We observe that while adversarial training in the feature space must deal with large and often unconstrained regions, UAPs in the problem space identify specific vulnerabilities that allow us to harden a classifier more effectively, shifting the challenges and associated cost of identifying new universal adversarial transformations back to the attacker.

show abstract

Section: Attack Scope and Objectivesmentioning

confidence: 99%

Realizable Universal Adversarial Perturbations for Malware

Labaca-Castro¹,

Muñoz-González²,

Pendlebury³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In Figure 3a, BadNets (No AWP) can be easily detected by the noise response method, while models with AWPs cannot be detected. Zhang et al (2020) propose to detect backdoors via targeted Universal Adversarial Perturbation (UAP) (Moosavi-Dezfooli et al, 2017), with the backdoor target as the target label. From Figure 3b, we can rank the detection difficulty as follows: Anchoring (AWP) > BadNets (AWP) > BadNets (No AWP).…”

Section: Backdoor Detection and Mitigationmentioning

confidence: 99%

Section: Backdoor Attackmentioning

confidence: 99%

“…Backdoor detection (Huang et al, 2020;Harikumar et al, 2020;Kwon, 2020;Zhang et al, 2020;Erichson et al, 2020) methods or backdoor mitigation methods (Yao et al, 2019;Zhao et al, 2020;Liu et al, 2018a) can be utilized to defend against backdoors. Backdoor detection methods usually identify the existence of backdoors in the model, via the responses of the model to input noises (Erichson et al, 2020) or universal adversarial perturbations (Zhang et al, 2020). Typical backdoor mitigation methods mitigate backdoors via fine-tuning, including direct fine-tuning models (Yao et al, 2019), fine-tuning after pruning (Liu et al, 2018a), and fine-tuning guided by knowledge distillation .…”

Section: Backdoor Attackmentioning

confidence: 99%

See 1 more Smart Citation

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

Zhang¹,

Lyu²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Since training a large-scale backdoored model from scratch requires a large training dataset, several recent attacks have considered to inject backdoors into a trained clean model without altering model behaviors on the clean data. Previous work finds that backdoors can be injected into a trained clean model with Adversarial Weight Perturbation (AWP). Here AWPs refers to the variations of parameters that are small in backdoor learning. In this work, we observe an interesting phenomenon that the variations of parameters are always AWPs when tuning the trained clean model to inject backdoors. We further provide theoretical analysis to explain this phenomenon. We formulate the behavior of maintaining accuracy on clean data as the consistency of backdoored models, which includes both global consistency and instance-wise consistency. We extensively analyze the effects of AWPs on the consistency of backdoored models. In order to achieve better consistency, we propose a novel anchoring loss to anchor or freeze the model behaviors on the clean data, with a theoretical guarantee. Both the analytical and the empirical results validate the effectiveness of the anchoring loss in improving the consistency, especially the instance-wise consistency.

show abstract

Potential cyber threats of adversarial attacks on autonomous driving models

Boltachev

2023

J Comput Virol Hack Tech

View full text Add to dashboard Cite

Cassandra: Detecting Trojaned Networks From Adversarial Perturbations

Cited by 10 publications

References 21 publications

Realizable Universal Adversarial Perturbations for Malware

Realizable Universal Adversarial Perturbations for Malware

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

Potential cyber threats of adversarial attacks on autonomous driving models

Contact Info

Product

Resources

About