MESDeceiver: Efficiently Generating Natural Language Adversarial Examples

Zhao, Tengfei; Ge, Zhaocheng; Hu, Hanping; Shi, Dingmeng

doi:10.1109/ijcnn55064.2022.9892490

Cited by 30 publications

(41 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conduct extensive experiments to verify our method on both image classification and semantic segmentation tasks. In Section 4.1, we first train classification models on ImageNet [12] and demonstrate that our models obtain significant improvement on various robustness benchmarks, including ImageNet-A [69], ImageNet-C [27], ImageNet-R [26], and ImageNet-P [27]. Then, in Section 4.2, we take our best pre-trained model and further finetune it on Cityscapes [11] for semantic segmentation.…”

Section: Methodsmentioning

confidence: 99%

“…Despite the success of vision transformers (ViTs), their performance still drops significantly on common image corruptions such as ImageNet-C [27,57], adversarial examples [20,18,44], and out-of-distribution examples as benchmarked in ImageNet-A/R/P [69,27]. In this paper, we examine a key component of ViTs, i.e., the self-attention mechanism, to understand these performance drops.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robustifying Token Attention for Vision Transformers

Guo¹,

Stutz²,

Schiele³

2023

Preprint

View full text Add to dashboard Cite

Despite the success of vision transformers (ViTs), they still suffer from significant drops in accuracy in the presence of common corruptions, such as noise or blur. Interestingly, we observe that the attention mechanism of ViTs tends to rely on few important tokens, a phenomenon we call token overfocusing. More critically, these tokens are not robust to corruptions, often leading to highly diverging attention patterns. In this paper, we intend to alleviate this overfocusing issue and make attention more stable through two general techniques: First, our Token-aware Average Pooling (TAP) module encourages the local neighborhood of each token to take part in the attention mechanism. Specifically, TAP learns average pooling schemes for each token such that the information of potentially important tokens in the neighborhood can adaptively be taken into account. Second, we force the output tokens to aggregate information from a diverse set of input tokens rather than focusing on just a few by using our Attention Diversification Loss (ADL). We achieve this by penalizing high cosine similarity between the attention vectors of different tokens. In experiments, we apply our methods to a wide range of transformer architectures and improve robustness significantly. For example, we improve corruption robustness on ImageNet-C by 2.4% while simultaneously improving accuracy by 0.4% based on state-of-the-art robust architecture FAN. Also, when finetuning on semantic segmentation tasks, we improve robustness on CityScapes-C by 2.4% and ACDC by 3.1%.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Robustifying Token Attention for Vision Transformers

Guo¹,

Stutz²,

Schiele³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Rob-GAN [20] introduced adversarial examples into GAN that not only could accelerate the training process by rapidly generating adversarial examples but also improve the quality of generating image and the robustness of discriminator. Zhao et al [21] introduced Natural GAN model which could search adversarial example vector in low-embedding latent space and generate more targeted and natural adversarial perturbations. Deb et al [22] focused on adversarial face synthesis and utilized human identity matching information to train GAN to gain face adversarial examples.…”

Section: Generator-based Adversarial Attacksmentioning

confidence: 99%

A Sparsity-Limitation-Based High-Dimensional Distribution Searching Algorithm for Adversarial Attack

et al. 2022

View full text Add to dashboard Cite

Deep neural networks-based image classification systems could suffer from adversarial attack algorithms, which generate input examples by adding deliberately crafted yet imperceptible noise to original inputs. To reduce the impact on human visual sense and to ensure adversarial attack ability, the input image needs to be modified by pixels in considerable iterations which is time consuming. By using sparse mapping network to map the input into a higher dimensional space, searching space of adversarial perturbation distribution is enlarged to better acquire perturbation information. Taking both searching speed and searching effectiveness into consideration, sparsity limitation is introduced to suppress unnecessary neurons during parameter updating process. Based on different eye sensitivity of different colors, maps of each color channel are disturbed by perturbations with different strengths to reduce visual perception. Numerical experiments confirm that compared with the state-of-the-art adversarial attack algorithms, the proposed SparseAdv performs a relatively high attack ability, better imperceptible visualization, and faster generation speed.

show abstract

“…The target distribution vector is computed by setting the target class to a specified value and renormalizing the remaining distribution so that other classes maintain the original relative order. The authors present two approaches for training the ATN.Zhao et al[67] use generative models to generate adversarial examples that look natural rather than noisy variants of existing data. The presented approach leverages GANs to sample images of the desired data distribution and induce perturbations in the latent space instead of the images.The intuition is that since G has been trained using the GAN procedure to generate realistic images from the noise vector z, the generator will map z to a natural-looking image independently of its value.…”

mentioning

confidence: 99%

“…Figure 2.8: Workflow for generating adversarial examples with WGAN as described in[67] (reproduced from[67]). …”

mentioning

confidence: 99%

GASTeN: Generative Adversarial Stress Test Networks

Cunha

Soares

Restivo

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Machine learning (ML) and deep learning (DL) are now ubiquitous in our society, and techniques that enable responsible usage are fundamental to safeguard people from being negatively affected. One particular example of DL's success is image classification. However, DL techniques function as black-box models whose knowledge representation is difficult to comprehend, and understanding the conditions under which they behave correctly is hard. Another example of a DL application is data generation, for which an algorithm known as GANs, mainly used for data augmentation, has achieved remarkable success. In GANs, two networks -the generator and the discriminator -are simultaneously trained. The generator learns to produce realistic data by trying to fool the discriminator, which is trained to distinguish between real and fake samples.This dissertation proposes a GAN-based approach for synthesizing new data to help understand DL image classifiers. We aim to generate examples that are hard for a given classifier that we could, ultimately, systematically analyze to get information about cases where the model's performance degrades. For that, we opt to generate data classified with low confidence by a classifier. Our approach, dubbed GASTeN, consists of modifying the loss function of the generator to include a new objective, dubbed confusion distance, which reflects how far the generated images are from having the desired output by the targetted classifier, i.e., the one we wish to evaluate. It introduces two hyperparameters, a weight to factor the new loss term, and the duration of pre-training of the GAN without any modifications.We empirically evaluate our proposal by instantiating it with a DCGAN architecture and a confusion distance suitable for binary classification. In our experiments, we target classifiers of binary subsets of the MNIST and Fashion MNIST datasets. We explore several hyperparameter configurations and target classifiers with different performances, analyzing the algorithm's behavior by collecting quantitative metrics for the two optimization objectives -FID for image realness and the average value of the confusion distance for the goal of confusing the classifier. In our experiments, we show scenarios in which we can obtain a generator with the desired properties of generating data with high realisticness that is mainly classified with low confidence by the target classifiers, along with scenarios where our goal is not attained. We conclude that, while challenging to optimize for both objectives simultaneously, it is possible to achieve images with the desired properties, albeit at the cost of carefully chosen hyperparameters.

show abstract

MESDeceiver: Efficiently Generating Natural Language Adversarial Examples

Cited by 30 publications

References 14 publications

Robustifying Token Attention for Vision Transformers

Robustifying Token Attention for Vision Transformers

A Sparsity-Limitation-Based High-Dimensional Distribution Searching Algorithm for Adversarial Attack

GASTeN: Generative Adversarial Stress Test Networks

Contact Info

Product

Resources

About