Bridge the Gap Between CV and NLP! An Optimization-based Textual Adversarial Attack Framework

Lifan, Yuan,; Zhang, Yichi; Yangyi, Chen,; Wei, Wei

doi:10.48550/arxiv.2110.15317

Cited by 6 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text-based model robustness. Recent works find that ML models are generally not robust in different NLP tasks [34,36,46,63,68,70,72]. In response, a plethora of studies have endeavored to enhance their robustness, such as adversarial training [18,29,52,75] and certified robustness [16,20,23,45,54,64,67].…”

Section: Related Workmentioning

confidence: 99%

Towards Robustness Analysis of E-Commerce Ranking System

Wang,

Huang,

Cheng

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Information retrieval (IR) is a pivotal component in various applications. Recent advances in machine learning (ML) have enabled the integration of ML algorithms into IR, particularly in ranking systems. While there is a plethora of research on the robustness of ML-based ranking systems, these studies largely neglect commercial e-commerce systems and fail to establish a connection between real-world and manipulated query relevance. In this paper, we present the first systematic measurement study on the robustness of e-commerce ranking systems. We define robustness as the consistency of ranking outcomes for semantically identical queries. To quantitatively analyze robustness, we propose a novel metric that considers both ranking position and item-specific information that are absent in existing metrics. Our large-scale measurement study with real-world data from e-commerce retailers reveals an open opportunity to measure and improve robustness since semantically identical queries often yield inconsistent ranking results. Based on our observations, we propose several solution directions to enhance robustness, such as the use of Large Language Models. Note that the issue of robustness discussed herein does not constitute an error or oversight. Rather, in scenarios where there exists a vast array of choices, it is feasible to present a multitude of products in various permutations, all of which could be equally appealing. However, this extensive selection may lead to customer confusion. As e-commerce retailers use various techniques to improve the quality of search results, we hope that this research offers valuable guidance for measuring the robustness of the ranking systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Robustness Analysis of E-Commerce Ranking System

Wang,

Huang,

Cheng

et al. 2024

Companion Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

show abstract

“…By manipulating the model gradients to induce misclassification behavior, these examples are generated, and have been shown to transfer across many different models [20]. Because AEs directly modify the input, they can not be directly generated with gradient manipulation on texts [33,39,12]. This is because nondifferential input text lookup in NLP models impedes the model gradient manipulation from reaching a legitimate word/words that could be used to generate AEs.…”

Section: Background and Related Workmentioning

confidence: 99%

TrojBits: A Hardware Aware Inference-Time Attack on Transformer-Based Language Models

Al Ghanim,

Santriaji,

Lou

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Transformer-based language models demonstrate exceptional performance in Natural Language Processing (NLP) tasks but remain susceptible to backdoor attacks involving hidden input triggers. Trojan injection via hardware bitflips presents a significant challenge for contemporary language models. However, previous research overlooks practical hardware considerations, such as DRAM and cache memory structures, resulting in unrealistic attacks that demand the manipulation of an excessive number of parameters and bits. In this paper, we present TrojBits, a novel approach requiring minimal bit-flips to effectively insert Trojans into real-world Transformer language model systems. This is achieved through a three-module framework designed to efficiently target Transformer-based language models, consisting of Vulnerable Parameters Ranking (VPR), Hardware-aware Attack Optimization (HAO), and Vulnerable Bits Pruning (VBP). Within the VPR module, we are the first to employ Gradient-guided Fisher information to identify the most susceptible Transformer parameters, specifically in the word embedding layer. The HAO module then redistributes these parameters across multiple triggers, conforming to hardware constraints by incorporating a regularization term in the trojan optimization methodology. Finally, the VBP module aims to reduce the number of bit-flips by discarding less significant bits. We evaluate TrojBits on two representative NLP models, BERT and XLNE, on three classification tasks (SST2, OffensEval, and AG’s News). Our results demonstrate that our TrojBits successfully achieves the inference-time attack with only 64 parameters out of 116 million and 90-bit flips while maintaining the model performance.

show abstract

“…Nevertheless, as noted earlier, gradients are still useful when they are applied using domain-specific constraints. For example, one can find local (wordlevel) perturbations that lead to a certain adversarial outcome, if the perturbations are restricted to welldefined semantic categories (e.g., "blue" can be perturbed to any other color name) (Sha, 2020;Guo et al, 2021;Yuan et al, 2021).…”

Section: Gpt Continuous Prompt D-pr Ojmentioning

confidence: 99%

Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts

Khashabi¹,

Lyu²,

Min³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Fine-tuning continuous prompts for target tasks has recently emerged as a compact alternative to full model fine-tuning. Motivated by these promising results, we investigate the feasibility of extracting a discrete (textual) interpretation of continuous prompts that is faithful to the problem they solve. In practice, we observe a "wayward" behavior between the task solved by continuous prompts and the nearest neighbor discrete projections of these prompts: One can find continuous prompts that solve a task while being projected to an arbitrary text (e.g., definition of a different or even a contradictory task) and simultaneously being within a very small (2%) margin of the best continuous prompt of the same size for the task. We provide intuitions behind this odd and surprising behavior, as well as extensive empirical analyses quantifying the effect of design choices. For instance, larger models exhibit higher waywardness, i.e, we can find prompts that more closely map to any arbitrary text with a smaller drop of accuracy. These findings have important implications relating to the difficulty of faithfully interpreting continuous prompts and their generalization across models and tasks, providing guidance for future progress in prompting language models.

show abstract

Bridge the Gap Between CV and NLP! An Optimization-based Textual Adversarial Attack Framework

Cited by 6 publications

References 21 publications

Towards Robustness Analysis of E-Commerce Ranking System

Towards Robustness Analysis of E-Commerce Ranking System

TrojBits: A Hardware Aware Inference-Time Attack on Transformer-Based Language Models

Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts

Contact Info

Product

Resources

About