Adversarial Defense Through Network Profiling Based Path Extraction

Qiu, Yuxian; Leng, Jingwen; Guo, Cong; Chen, Quan; Li, Chao; Guo, Minyi; Zhu, Yuhao

doi:10.1109/cvpr.2019.00491

Cited by 47 publications

(29 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later, they detect adversarial examples by identifying inconsistencies between the original and the attribute-steered models. In [390], the authors proposed a mechanism to trace the activation paths of clean and adversarial images and detect adversarial perturbations based on the different characteristics of these paths. Liu et al [391] proposed to detect adversarial examples by analysing inputs from steganography point of view.…”

Section: B Detection For Defensementioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

Deep learning is at the heart of the current rise of artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently lead to a large influx of contributions in this direction. This article presents the first comprehensive survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.

show abstract

Section: B Detection For Defensementioning

confidence: 99%

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

2018

View full text Add to dashboard Cite

show abstract

“…FeatureMap is a naive baseline that uses the feature maps of convolutional layers as the inputs of the classifier. EffectivePath is a more advanced baseline that uses the effective path generated by Qiu et al [57] to train the classifier. The experiments were conducted on ResNet10 and the CIFAR-10 dataset (image size 32×32).…”

Section: Discussionmentioning

confidence: 99%

Dynamic slicing for deep neural networks

Zhang

Guo

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Program slicing has been widely applied in a variety of software engineering tasks. However, existing program slicing techniques only deal with traditional programs that are constructed with instructions and variables, rather than neural networks that are composed of neurons and synapses. In this paper, we propose NNSlicer, the first approach for slicing deep neural networks based on data flow analysis. Our method understands the reaction of each neuron to an input based on the difference between its behavior activated by the input and the average behavior over the whole dataset. Then we quantify the neuron contributions to the slicing criterion by recursively backtracking from the output neurons, and calculate the slice as the neurons and the synapses with larger contributions. We demonstrate the usefulness and effectiveness of NNSlicer with three applications, including adversarial input detection, model pruning, and selective model protection. In all applications, NNSlicer significantly outperforms other baselines that do not rely on data flow analysis. CCS CONCEPTS• Computing methodologies → Neural networks; • Software and its engineering → Dynamic analysis.

show abstract

“…With such taxonomy, we further apply analytic experiments to explore the function of each behavior according to distance. We are also looking forward to further analysis the behavior and function of variant patterns with probing task datasets (Conneau et al, 2018) and analytic tools (Qiu et al, 2019;Gan et al, 2020) as our next plan. Besides, there are several recent works focusing on the optimization of over-parameterized MHA mechanism (Michel et al, 2019;Kovaleva et al, 2019;.…”

Section: Discussionmentioning

confidence: 99%

How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT’s Attention

Guan¹,

Leng²,

Li³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Recent research on the multi-head attention mechanism, especially that in pre-trained models such as BERT, has shown us heuristics and clues in analyzing various aspects of the mechanism. As most of the research focus on probing tasks or hidden states, previous works have found some primitive patterns of attention head behavior by heuristic analytical methods, but a more systematic analysis specific on the attention patterns still remains primitive. In this work, we clearly cluster the attention heatmaps into significantly different patterns through unsupervised clustering on top of a set of proposed features, which corroborates with previous observations. We further study their corresponding functions through analytical study. In addition, our proposed features can be used to explain and calibrate different attention heads in Transformer models.

show abstract

Adversarial Defense Through Network Profiling Based Path Extraction

Cited by 47 publications

References 17 publications

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

Dynamic slicing for deep neural networks

How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT’s Attention

Contact Info

Product

Resources

About