TextTricker: Loss-based and gradient-based adversarial attacks on text classification models

Xu, Jincheng; Du, Qingfeng

doi:10.1016/j.engappai.2020.103641

Cited by 25 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we propose a scoring mechanism to identify the important tokens in a document which have a high impact on the final ranking result. Following [26,55], we first calculate the gradient magnitude with respect to each input unit. Then, we sum up the score for each dimension in the embedding space as the token-level importance score.…”

Section: Token Importance Rankingmentioning

confidence: 99%

PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models

Chen¹,

Zhang²,

Guo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural ranking models (NRMs) have shown remarkable success in recent years, especially with pre-trained language models. However, deep neural models are notorious for their vulnerability to adversarial examples. Adversarial attacks may become a new type of web spamming technique given our increased reliance on neural information retrieval models. Therefore, it is important to study potential adversarial attacks to identify vulnerabilities of NRMs before they are deployed.In this paper, we introduce the Adversarial Document Ranking Attack (ADRA) task against NRMs, which aims to promote a target document in rankings by adding adversarial perturbations to its text. We focus on the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire the rank positions of the partial retrieved list by querying the target model. This attack setting is realistic in real-world search engines. We propose a novel Pseudo Relevancebased ADversarial ranking Attack method (PRADA) that learns a surrogate model based on Pseudo Relevance Feedback (PRF) to generate gradients for finding the adversarial perturbations.Experiments on two web search benchmark datasets show that PRADA can outperform existing attack strategies and successfully fool the NRM with small indiscernible perturbations of text. CCS CONCEPTS• Information systems → Retrieval models and ranking; Adversarial retrieval.

show abstract

Section: Token Importance Rankingmentioning

confidence: 99%

PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models

Chen¹,

Zhang²,

Guo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In NLP, Current gradient-based methods do not generate words directly through continuous perturbations, but use first-order approximation to enumerate substitution words (Cheng et al, 2019;Behjati et al, 2019;Xu and Du, 2020). This one-off approach may result in large step size perturbation, violating the hypothesis of local linearization.…”

Section: Perturbationmentioning

confidence: 99%

“…And the methods that directly employ the gradients to craft adversarial samples is not applicable in NLP. Current practices of textual adversarial attacks that employ first-order approximation to find substitute words are less effective for one-off searching and can violate the local linearization assumption (Cheng et al, 2019;Behjati et al, 2019;Xu and Du, 2020).…”

Section: Introductionmentioning

confidence: 99%

Bridge the Gap Between CV and NLP! An Optimization-based Textual Adversarial Attack Framework

Lifan¹,

Zhang²,

Yangyi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite great success on many machine learning tasks, deep neural networks are still vulnerable to adversarial samples. While gradientbased adversarial attack methods are wellexplored in the field of computer vision, it is impractical to directly apply them in natural language processing due to the discrete nature of text. To bridge this gap, we propose a general framework to adapt existing gradientbased method to craft textual adversarial samples. In this framework, gradient-based continuous perturbations are added to the embedding layer and are amplified in the forward propagation process. Then the final perturbed latent representations are decoded with a mask language model head to obtain potential adversarial samples. In this paper, we instantiate our framework with Textual Projected Gradient Descent (TPGD). We conduct comprehensive experiments to evaluate our framework by performing transfer black-box attacks on BERT, RoBERTa and ALBERT on three benchmark datasets. Experimental results demonstrate our method achieves an overall better performance and produces more fluent and grammatical adversarial samples compared to strong baseline methods. All the code and data will be made public.

show abstract

“…The most well-known robustness attack methods include Fast Gradient Sign [31], Jacobian-based saliency map approach [34], C&W attack [7], etc. In recent years a great number of works like [19,9,59,37,18,62] have developed adversarial attack with a variety of methods and techniques.…”

Section: Adversarial Attacksmentioning

confidence: 99%

Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning

Li¹,

Yang²,

Huang³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper proposes a black box based approach for analysing deep neural networks (DNNs). We view a DNN as a function f from inputs to outputs, and consider the local robustness property for a given input. Based on scenario optimization technique in robust control design, we learn the score difference function fi − f ℓ with respect to the target label ℓ and attacking label i. We use a linear template over the input pixels, and learn the corresponding coefficients of the score difference function, based on a reduction to a linear programming (LP) problems. To make it scalable, we propose optimizations including components based learning and focused learning. The learned function offers a probably approximately correct (PAC) guarantee for the robustness property. Since the score difference function is an approximation of the local behaviour of the DNN, it can be used to generate potential adversarial examples, and the original network can be used to check whether they are spurious or not. Finally, we focus on the input pixels with large absolute coefficients, and use them to explain the attacking scenario. We have implemented our approach in a prototypical tool DeepPAC. Our experimental results show that our framework can handle very large neural networks like ResNet152 with 6.5M neurons, and often generates adversarial examples which are very close to the decision boundary.

show abstract

TextTricker: Loss-based and gradient-based adversarial attacks on text classification models

Cited by 25 publications

References 15 publications

PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models

PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models

Bridge the Gap Between CV and NLP! An Optimization-based Textual Adversarial Attack Framework

Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning

Contact Info

Product

Resources

About