A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification

Li, Xiangci; Burns, Gully; Peng, Nanyun

doi:10.48550/arxiv.2012.14500

Cited by 10 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…USE-QA q is a universal sentence encoder [71] model fine-tuned on the SQuAD [94] dataset. Paragraph-Joint [95] uses the BioSentVec [96] model for the retrieval purpose. BioSentVec is a sent2vec [97]…”

Section: Baselines Usedmentioning

confidence: 99%

Improved Methods to Aid Unsupervised Evidence-Based Fact Checking for Online Heath News

Deka¹,

Jurek-Loughrey²,

Deepak³

2022

JDI

View full text Add to dashboard Cite

False information in the domain of online health related articles is of great concern, which has been witnessed abundantly in the current pandemic situation of Covid-19. Recent advancements in the field of Machine Learning and Natural Language Processing can be leveraged to aid people in distinguishing false information from the truth in the domain of online health articles. Whilst there has been substantial progress in this space over the years, research in this area has mainly focused on the sphere of political news. Health fake news is markedly different from fake news in the political context as health information should be evaluated against the most recent and reliable medical resources such as scholarly repositories. However, one of the challenges with such an approach is the retrieval of the pertinent resources. In this work, we formulate two techniques for the retrieval of the most relevant authoritative and reliable medical content from scholarly repositories which can be used to assess veracity of an online health article. The first technique is an unsupervised method of generating queries from claims which are extracted from an online health article. We propose a three-step approach for it and illustrate that our method is able to generate effective queries which can be used for retrieval of information from medical knowledge databases. The second method involves a filtering approach for extracting the most relevant information for the claims. We show how this can be achieved with the help of state of the art transformer models and illustrate it's effectiveness over other methods.

show abstract

Section: Baselines Usedmentioning

confidence: 99%

Improved Methods to Aid Unsupervised Evidence-Based Fact Checking for Online Heath News

Deka¹,

Jurek-Loughrey²,

Deepak³

2022

JDI

View full text Add to dashboard Cite

show abstract

“…Setup For SCIFACT, we chose three systems for testing our attack: VeriSci (Wadden et al 2020), ParagraphJoint (Li, Burns, and Peng 2021), and SciKGAT (Liu et al 2020a). The VeriSci model was released by the creators of the SCIFACT benchmark and retrieves relevant abstracts to a claim using TF-IDF.…”

Section: Scifact Studymentioning

confidence: 99%

Synthetic Disinformation Attacks on Automated Fact Verification Systems

Bosselut

Manning

2022

AAAI

View full text Add to dashboard Cite

Automated fact-checking is a needed technology to curtail the spread of online misinformation. One current framework for such solutions proposes to verify claims by retrieving supporting or refuting evidence from related textual sources. However, the realistic use cases for fact-checkers will require verifying claims against evidence sources that could be affected by the same misinformation. Furthermore, the development of modern NLP tools that can produce coherent, fabricated content would allow malicious actors to systematically generate adversarial disinformation for fact-checkers. In this work, we explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings: ADVERSARIAL ADDITION, where we fabricate documents and add them to the evidence repository available to the fact-checking system, and ADVERSARIAL MODIFICATION, where existing evidence source documents in the repository are automatically altered. Our study across multiple models on three benchmarks demonstrates that these systems suffer significant performance drops against these attacks. Finally, we discuss the growing threat of modern NLG systems as generators of disinformation in the context of the challenges they pose to automated fact-checkers.

show abstract

“…When dealing with claim verification, most recent systems fine-tune a large pre-trained language model to do three-way label prediction, including VERISCI ( Wadden et al, 2020 ), VERT5ERINI ( Pradeep et al, 2020 ), and ParagraphJoint ( Li, Burns & Peng, 2021 ). Despite the evident effectiveness of these methods, fine-tuning models depends on the availability of substantial amounts of labelled data, which are not always accessible, particularly for new domains.…”

Section: Introductionmentioning

confidence: 99%

Aggregating pairwise semantic differences for few-shot claim verification

Zeng¹,

Zubiaga²

2022

PeerJ Computer Science

View full text Add to dashboard Cite

As part of an automated fact-checking pipeline, the claim verification task consists in determining if a claim is supported by an associated piece of evidence. The complexity of gathering labelled claim-evidence pairs leads to a scarcity of datasets, particularly when dealing with new domains. In this article, we introduce Semantic Embedding Element-wise Difference (SEED), a novel vector-based method to few-shot claim verification that aggregates pairwise semantic differences for claim-evidence pairs. We build on the hypothesis that we can simulate class representative vectors that capture average semantic differences for claim-evidence pairs in a class, which can then be used for classification of new instances. We compare the performance of our method with competitive baselines including fine-tuned Bidirectional Encoder Representations from Transformers (BERT)/Robustly Optimized BERT Pre-training Approach (RoBERTa) models, as well as the state-of-the-art few-shot claim verification method that leverages language model perplexity. Experiments conducted on the Fact Extraction and VERification (FEVER) and SCIFACT datasets show consistent improvements over competitive baselines in few-shot settings. Our code is available.

show abstract

A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification

Cited by 10 publications

References 17 publications

Improved Methods to Aid Unsupervised Evidence-Based Fact Checking for Online Heath News

Improved Methods to Aid Unsupervised Evidence-Based Fact Checking for Online Heath News

Synthetic Disinformation Attacks on Automated Fact Verification Systems

Aggregating pairwise semantic differences for few-shot claim verification

Contact Info

Product

Resources

About