Improving Black-box Adversarial Attacks with a Transfer-based Prior

Cheng, Shuyu; Dong, Yinpeng; Pang, Tianyu; Su, Hang; Zhu, Jun

doi:10.48550/arxiv.1906.06919

Cited by 10 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later, they improved the attack by drastically reducing the number of queries by focusing on estimating gradient signs instead of gradients [83]. In another attempt to decrease the number of queries, Cheng et al [85] also introduced a priorguided random gradient-free method.…”

Section: B Black-box Attacksmentioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

Deep learning is at the heart of the current rise of artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently lead to a large influx of contributions in this direction. This article presents the first comprehensive survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.

show abstract

Section: B Black-box Attacksmentioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

show abstract

“…This is straightforward since we can utilize {u i } q i=1 which is uniformly sampled from the (d − 1)-dimensional space H. Proposition 10. For any [9] for the proof.). Therefore,…”

Section: B54 Proof Of Lemma 12mentioning

confidence: 99%

“…We found that in this task, ARS-based methods perform comparably to RGF-based ones. Since it has been shown that PRGF with transfer-based prior works well [9], we only report the results of RGF and History-PRGF in Tab. 1, where the subscript of the method name indicates the learning rate.…”

Section: Black-box Adversarial Attacksmentioning

confidence: 99%

“…To improve, various attempts have been made to augment random search with (extra) prior information. For instance, [16,23] use a time-dependent prior (i.e., the gradient estimated in the last step), while [20,9,5] use surrogate gradients obtained from other sources. 2 Among these methods, [16,20,9,23] propose an objective function for designing a good gradient estimator to justify their choice or to choose optimal hyperparameters in the gradient estimator; and [23] uses a subspace estimator that maximizes the squared cosine similarity with the true gradient and finds a better descent direction than both the prior direction and the randomly sampled direction.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, [16,23] use a time-dependent prior (i.e., the gradient estimated in the last step), while [20,9,5] use surrogate gradients obtained from other sources. 2 Among these methods, [16,20,9,23] propose an objective function for designing a good gradient estimator to justify their choice or to choose optimal hyperparameters in the gradient estimator; and [23] uses a subspace estimator that maximizes the squared cosine similarity with the true gradient and finds a better descent direction than both the prior direction and the randomly sampled direction. However, all these methods treat gradient estimation and the optimization algorithm separately, and it remains unclear whether these gradient estimators and the corresponding prior-exploiting optimization methods are theoretically sound, and what role the prior plays in accelerating convergence.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms

Cheng¹,

Wu²,

Zhu³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Zeroth-order (ZO) optimization is widely used to handle challenging tasks, such as query-based black-box adversarial attacks and reinforcement learning. Various attempts have been made to integrate prior information into the gradient estimation procedure based on finite differences, with promising empirical results. However, their convergence properties are not well understood. This paper makes an attempt to fill this gap by analyzing the convergence of prior-guided ZO algorithms under a greedy descent framework with various gradient estimators. We provide a convergence guarantee for the prior-guided random gradient-free (PRGF) algorithms. Moreover, to further accelerate over greedy descent methods, we present a new accelerated random search (ARS) algorithm that incorporates prior information, together with a convergence analysis. Finally, our theoretical results are confirmed by experiments on several numerical benchmarks as well as adversarial attacks. Our code is available at https://github.com/csy530216/pg-zoo.

show abstract

Adversarial Transfer Attacks With Unknown Data and Class Overlap

Richards

Nguyen

Capps

et al. 2021

Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security

View full text Add to dashboard Cite

The ability to transfer adversarial attacks from one model (the surrogate) to another model (the victim) has been an issue of concern within the machine learning (ML) community. The ability to successfully evade unseen models represents an uncomfortable level of ease toward implementing attacks. In this work we note that as studied, current transfer attack research has an unrealistic advantage for the attacker: the attacker has the exact same training data as the victim. We present the first study of transferring adversarial attacks focusing on the data available to attacker and victim under imperfect settings without querying the victim, where there is some variable level of overlap in the exact data used or in the classes learned by each model. This threat model is relevant to applications in medicine, malware, and others. Under this new threat model attack success rate is not correlated with data or class overlap in the way one would expect, and varies with dataset. This makes it difficult for attacker and defender to reason about each other and contributes to the broader study of model robustness and security. We remedy this by developing a masked version of Projected Gradient Descent that simulates class disparity, which enables the attacker to reliably estimate a lower-bound on their attack's success.

show abstract

Improving Black-box Adversarial Attacks with a Transfer-based Prior

Cited by 10 publications

References 16 publications

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms

Adversarial Transfer Attacks With Unknown Data and Class Overlap

Contact Info

Product

Resources

About