Adversarial Texts with Gradient Methods

Gong, Zhitao; Wang, Wenlu; Li, Bo; Song, Dawn; Ku, Wei‐Shinn

doi:10.48550/arxiv.1801.07175

Cited by 34 publications

(55 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from CV where the image is represented as continuous data, the main problem of generating adversarial text in NLP is the discrete input space. To address the problem, Gong et al [22] proposed to attack in the embedding space. They used FGSM to produce perturbations in word embedding and used nearest neighbor search to find the closest words.…”

Section: Adversarial Operationmentioning

confidence: 99%

Are Neural Ranking Models Robust?

Chen¹,

Zhang²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, we have witnessed the bloom of neural ranking models in the information retrieval (IR) field. So far, much effort has been devoted to developing effective neural ranking models that can generalize well on new data. There has been less attention paid to the robustness perspective. Unlike the effectiveness which is about the average performance of a system under normal purpose, robustness cares more about the system performance in the worst case or under malicious operations instead. When a new technique enters into the real-world application, it is critical to know not only how it works in average, but also how would it behave in abnormal situations. So we raise the question in this work: Are neural ranking models robust? To answer this question, firstly, we need to clarify what we refer to when we talk about the robustness of ranking models in IR. We show that robustness is actually a multi-dimensional concept and there are three ways to define it in IR: 1) The performance variance under the independent and identically distributed (I.I.D.) setting; 2) The out-of-distribution (OOD) generalizability; and 3) The defensive ability against adversarial operations. The latter two definitions can be further specified into two different perspectives respectively, leading to 5 robustness tasks in total. Based on this taxonomy, we build corresponding benchmark datasets, design empirical experiments, and systematically analyze the robustness of several representative neural ranking models against traditional probabilistic ranking models and learning-to-rank (LTR) models. The empirical results show that there is no simple answer to our question. While neural ranking models are less robust against other IR models in most cases, some of them can still win 1 out of 5 tasks. This is the first comprehensive study on the robustness of neural ranking models. We believe the way we study the robustness as well as our findings would be beneficial to the IR community. We will also release all the data and codes to facilitate the future research in this direction.CCS Concepts: • Information systems → Retrieval models and ranking.

show abstract

Section: Adversarial Operationmentioning

confidence: 99%

Are Neural Ranking Models Robust?

Chen¹,

Zhang²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The GloVe learning technique has been utilized by numerous research studies [24], [31], [39], [42], [51] to study the impact of word learning representation on robustness. For instance, in [24], Rychalska et al introduced WildNLP, a framework for studying model stability with text corruptions, e.g., keyboard errors and misspelling.…”

Section: Embedding Techniquesmentioning

confidence: 99%

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Omar¹,

Choi²,

Nyang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent natural language processing (NLP) techniques have accomplished high performance on benchmark datasets, primarily due to the significant improvement in the performance of deep learning. The advances in the research community have led to great enhancements in state-of-the-art production systems for NLP tasks, such as virtual assistants, speech recognition, and sentiment analysis. However, such NLP systems still often fail when tested with adversarial attacks. The initial lack of robustness exposed troubling gaps in current models' language understanding capabilities, creating problems when NLP systems are deployed in real life. In this paper, we present a structured overview of NLP robustness research by summarizing the literature in a systemic way across various dimensions. We then take a deep-dive into the various dimensions of robustness, across techniques, metrics, embeddings, and benchmarks. Finally, we argue that robustness should be multi-dimensional, provide insights into current research, identify gaps in the literature to suggest directions worth pursuing to address these gaps.

show abstract

“…To address the issue of embedding-to-word mapping, Gong et al (2018) propose to use nearestneighbour search to find the closest words to the perturbed embeddings. However, this method treats all tokens as equally vulnerable and replace all tokens with their nearest neighbours, which leads to non-sensical, word-salad outputs.…”

Section: Related Workmentioning

confidence: 99%

Grey-box Adversarial Attack And Defence For Sentiment Classification

Zhong

Yepes

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a grey-box adversarial attack and defence framework for sentiment classification. We address the issues of differentiability, label preservation and input reconstruction for adversarial attack and defence in one unified framework. Our results show that once trained, the attacking model is capable of generating high-quality adversarial examples substantially faster (one order of magnitude less in time) than state-of-the-art attacking methods. These examples also preserve the original sentiment according to human evaluation. Additionally, our framework produces an improved classifier that is robust in defending against multiple adversarial attacking methods.

show abstract

Adversarial Texts with Gradient Methods

Cited by 34 publications

References 0 publications

Are Neural Ranking Models Robust?

Are Neural Ranking Models Robust?

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Grey-box Adversarial Attack And Defence For Sentiment Classification

Contact Info

Product

Resources

About