Evaluating adversarial attacks against multiple fact verification systems

Thorne, James; Vlachos, Andreas; Christodoulopoulos, Christos; Mittal, Arpit

doi:10.18653/v1/d19-1292

Cited by 31 publications

(28 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While much recent work in adversarial attacks aims to break NLI systems and is especially adapted to this problem [13,29], these stress tests have been applied to several other tasks, e.g. Question-Answering [49], Machine Translation [4], or Fact Checking [1,44]. Unfortunately, preserving the semantics of a sentence while automatically generating these adversarial attacks is difficult, which is why some works have defined small stress tests manually [19,27].…”

Section: Datasetmentioning

confidence: 99%

“…To measure the effectiveness of each adversarial attack a ∈ A , we calculate the potency score introduced by Thorne et al [44] as the average reduction from a perfect score and across the systems s ∈ S: with c a representing the transformation correctness from test to adversarial samples and a function f that returns the performance score for a system s on an adversarial attack set a.…”

Section: Metrics (Original)mentioning

confidence: 99%

“…We then probe the robustness of our models via three adversarial attacks (perturbations of the original test sets of all StD datasets used) and measure it with a modified version of the Resilience score by Thorne et al [44]. However, while we gain significant performance improvements on the original test sets by using TL and MDL, the expected robustness of these approaches is missing.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Stance Detection Benchmark: How Robust is Your Stance Detection?

2021

View full text Add to dashboard Cite

Stance detection (StD) aims to detect an author’s stance towards a certain topic and has become a key component in applications like fake news detection, claim validation, or argument search. However, while stance is easily detected by humans, machine learning (ML) models are clearly falling short of this task. Given the major differences in dataset sizes and framing of StD (e.g. number of classes and inputs), ML models trained on a single dataset usually generalize poorly to other domains. Hence, we introduce a StD benchmark that allows to compare ML models against a wide variety of heterogeneous StD datasets to evaluate them for generalizability and robustness. Moreover, the framework is designed for easy integration of new datasets and probing methods for robustness. Amongst several baseline models, we define a model that learns from all ten StD datasets of various domains in a multi-dataset learning (MDL) setting and present new state-of-the-art results on five of the datasets. Yet, the models still perform well below human capabilities and even simple perturbations of the original test samples (adversarial attacks) severely hurt the performance of MDL models. Deeper investigation suggests overfitting on dataset biases as the main reason for the decreased robustness. Our analysis emphasizes the need of focus on robustness and de-biasing strategies in multi-task learning approaches. To foster research on this important topic, we release the dataset splits, code, and fine-tuned weights.

show abstract

Section: Datasetmentioning

confidence: 99%

Section: Metrics (Original)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Stance Detection Benchmark: How Robust is Your Stance Detection?

2021

View full text Add to dashboard Cite

show abstract

“…The submissions were scored using 'potency' and 'resilience' (Thorne et al, 2019) that compute a weighted average of FEVER scores: accounting for the correctness of adversarial instances.…”

Section: Scoring Methodsmentioning

confidence: 99%

The FEVER2.0 Shared Task

Thorne¹,

Vlachos²,

Cocarascu³

et al. 2019

Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)

Self Cite

View full text Add to dashboard Cite

We present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. The task challenged participants to both build systems to verify factoid claims using evidence retrieved from Wikipedia and to generate adversarial attacks against other participant's systems. The shared task had three phases: building, breaking and fixing. There were 8 systems in the builder's round, three of which were new qualifying submissions for this shared task, and 5 adversaries generated instances designed to induce classification errors and one builder submitted a fixed system which had higher FEVER score and resilience than their first submission. All but one newly submitted systems attained FEVER scores higher than the best performing system from the first shared task and under adversarial evaluation, all systems exhibited losses in FEVER score. There was a great variety in adversarial attack types as well as the techniques used to generate the attacks, In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.

show abstract

“…These rule-based adversarial attacks are more common in the written vernacular and therefore are often found in misinformation posts on social media. [3].…”

Section: Evidence-based Overreliance -mentioning

confidence: 99%

The Accidental Checkmate: Understanding the Intent behind sharing Misinformation on Social Media

Agarwal¹

2020

Preprint

View full text Add to dashboard Cite

The bloom of COVID19 has resulted in the explosion of ripple pollens which have severely affected the world community in the terms of their multi-axial impact. These pollens, despite being indistinguishable, have a varied set of characteristics in terms of their origin and contribution towards the overall declining homeostasis of human beings. The most prominent of these pollens are misinformation. Various studies have been conducted, performed, and stochastically replicated to build ML-based models to accurately detect misinformation and its variates on the common modalities of spread. However, the recent independent analysis conducted on the prior studies reveals how the current fact-checking systems fail and fall flat in fulfilling any practical demands that the misinfodemic of COVID19 brought for us. While the scientific community broadly accepts the pandemic-like resemblance of the rampant misinformation spread, we must also make sure that our response to the same is multi-faceted, interdisciplinary, and doesn’t stand restricted. As crucial it is to chart the features of misinformation spread, it is also important to understand why it spreads in the first place? Our paper deals with the latter question through a game-theory based approach. We implement a game with two social media users or players who aim at increasing their outreach on their social media handles whilst spreading misinformation knowingly. We take five independent parameters from 100 Twitter handles that have shared misinformation during the period of COVID19. Twitter was chosen as it is a prominent social media platform accredited to the major modality for misinformation spread. The outreach increment on the user’s Twitter handles were measured using various features provided by Twitter - number of comments, number of retweets, and number of likes. Later, using a computational neuroscientific approach, we map each of these features with the type of neural system they trigger in a person’s brain. This helps in understanding how misinformation whilst being used as an intentional decoy to increase outreach on social media, also, affects the human social cognition system eliciting pseudo-responses that weren’t intended otherwise leading to realizing possible neuroscientific correlation as to how spreading misinformation on social media intentionally/unintentionally becomes a strategic maneuver to increased reach and possibly a false sense of accomplishment.

show abstract

Evaluating adversarial attacks against multiple fact verification systems

Cited by 31 publications

References 22 publications

Stance Detection Benchmark: How Robust is Your Stance Detection?

Stance Detection Benchmark: How Robust is Your Stance Detection?

The FEVER2.0 Shared Task

The Accidental Checkmate: Understanding the Intent behind sharing Misinformation on Social Media

Contact Info

Product

Resources

About