Generating Label Cohesive and Well-Formed Adversarial Claims

Abstract

Adversarial attacks reveal important vulnerabilities and flaws of trained models. One potent type of attack are universal adversarial triggers, which are individual n-grams that, when appended to instances of a class under attack, can trick a model into predicting a target class. However, for inference tasks such as fact checking, these triggers often inadvertently invert the meaning of instances they are inserted in. In addition, such attacks produce semantically nonsensical inputs, as they simply concatenate triggers to existing samples. Here, we investigate how to generate adversarial attacks against fact checking systems that preserve the ground truth meaning and are semantically valid. We extend the HotFlip attack algorithm used for universal trigger generation by jointly minimizing the target class loss of a fact checking model and the entailment class loss of an auxiliary natural language inference model. We then train a conditional language model to generate semantically valid statements, which include the found universal triggers. We find that the generated attacks maintain the directionality and semantic validity of the claim better than previous work. * denotes equal contribution Dissociative disorders have been attributed to disruptions in memory caused by trauma or other forms of stress. SUPPORTS REFUTESDissociative identity disorder, or DID, may be the result of memory disruptions that have been induced by psychological trauma. CLAIMdon,already,more,during,home GPT-2 Claim GenerationDissociative disorders have been attributed to disrupted brain activity during trauma or other forms of stress.

Stance Detection Benchmark: How Robust is Your Stance Detection?

“…The inter-annotator agreement (computed with Cohen's kappa [7]) between the annotators is 0.47, which signals "moderate" agreement [24]. This is comparable to the inter-annotator agreement in Atanasova et al [1], where claims generated with GPT-2 were annotated for semantic coherence. Table 10 shows the Cohen's kappa for each dataset separately.…”

“…While much recent work in adversarial attacks aims to break NLI systems and is especially adapted to this problem [13,29], these stress tests have been applied to several other tasks, e.g. Question-Answering [49], Machine Translation [4], or Fact Checking [1,44]. Unfortunately, preserving the semantics of a sentence while automatically generating these adversarial attacks is difficult, which is why some works have defined small stress tests manually [19,27].…”

“…Due to the high subjectivity of this task, the annotation was conducted by two human annotators; the first author and a postdoctoral researcher with background in natural language processing (not involved in this work). The inter-annotator agreement was computed with Cohen's Kappa [7] and signals "moderate" agreement [24] with = 0.47 (see Appendix 2 for more information about the annotation process), which is comparable to the inter-annotator agreement in Atanasova et al [1], where claims generated with GPT-2 were annotated for semantic coherence. The percentage of samples annotated as "semantically equivalent" is 48.4% (average of both annotators), resulting in a correctness ratio c a of 0.484 for the paraphrase attack.…”

“…While these differences are problematic for cross-domain performance, it can also be seen as an advantage, as it concludes in an abundance of datasets from different domains that can be integrated into transfer or multi-task learning approaches. Yet, given the decent human performance on this task, it is hard to grasp why ML models fall short of StD, while they are almost on par for related tasks like Sentiment Analysis 1 and Natural Language Inference 2 (NLI).…”

“…The contributions of this paper are as follows: (1) to the best of our knowledge, within the field of StD we are the first to combine learning from related tasks (via TL) and MDL, designed to capture all facets of StD tasks, and achieve new state-of-the-art results on five of ten datasets. (2) In an indepth analysis with adversarial attacks, we show that TL and MDL for StD generally improves the performance of ML models, but also drastically reduces their robustness if compared to SDL models.…”