Extractive and Abstractive Explanations for Fact-Checking and Evaluation of News

Kazemi, Ashkan; Li, Zehua; Pérez-Rosas, Verónica; Mihalcea, Rada

doi:10.18653/v1/2021.nlp4if-1.7

Cited by 8 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the training data of the GPT models includes vast amounts of web data, including a filtered version of the Common Crawl Corpus (Brown et al, 2020), there is a significant risk of data leakage for the task of fact-checking. Previous research has shown that misinformation does not exist in isolation, but is repeated (Shaar et al, 2020) across platforms (Micallef et al, 2022) and languages (Kazemi et al, 2022;Quelle et al, 2023). As misinformation is repeated and re-occurs, the ability of models to retain previously fact-checked claims can potentially be seen as a benefit rather than a drawback.…”

Section: Discussionmentioning

confidence: 99%

The perils and promises of fact-checking with large language models

Quelle,

Bovet

2024

Front. Artif. Intell.

View full text Add to dashboard Cite

Automated fact-checking, using machine learning to verify claims, has grown vital as misinformation spreads beyond human fact-checking capacity. Large language models (LLMs) like GPT-4 are increasingly trusted to write academic papers, lawsuits, and news articles and to verify information, emphasizing their role in discerning truth from falsehood and the importance of being able to verify their outputs. Understanding the capacities and limitations of LLMs in fact-checking tasks is therefore essential for ensuring the health of our information ecosystem. Here, we evaluate the use of LLM agents in fact-checking by having them phrase queries, retrieve contextual data, and make decisions. Importantly, in our framework, agents explain their reasoning and cite the relevant sources from the retrieved context. Our results show the enhanced prowess of LLMs when equipped with contextual information. GPT-4 outperforms GPT-3, but accuracy varies based on query language and claim veracity. While LLMs show promise in fact-checking, caution is essential due to inconsistent accuracy. Our investigation calls for further research, fostering a deeper comprehension of when agents succeed and when they fail.

show abstract

Section: Discussionmentioning

confidence: 99%

The perils and promises of fact-checking with large language models

Quelle,

Bovet

2024

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…The sentences or sentence parts supporting the decision were highlighted, and users were asked to mark whether they agreed or disagreed with the highlighted evidence. Instead of relying on the weights of a model for explanation, Atanasova, Simonsen, Lioma, and Augenstein (2020) , Kotonya and Toni (2020) and Kazemi, Li, Pérez-Rosas, and Mihalcea (2021) applied extractive and abstractive summarisation techniques to provide users with justifications in natural language. Regarding bias mitigation, several mitigation methods have been proposed ( Hovy & Prabhumoye, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

Preventing profiling for ethical fake news detection

Allein

Moens

Perrotta

2023

Information Processing & Management

View full text Add to dashboard Cite

“…While fully automatic fake news detection and fact-checking systems (Pérez-Rosas et al 2018;Thorne and Vlachos 2018) remain an active research topic within the NLP community, there have been new research fronts in the fight against misinformation, including claim matching (Shaar et al 2020;Kazemi et al 2021a), check-worthiness detection (Hassan et al 2017;Konstantinovskiy et al 2021), explanations (Kazemi et al 2021b;…”

Section: Related Workmentioning

confidence: 99%

Matching Tweets With Applicable Fact-Checks Across Languages

Kazemi¹,

Li²,

Pérez-Rosas³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

An important challenge for news fact-checking is the effective dissemination of existing fact-checks. This in turn brings the need for reliable methods to detect previously factchecked claims. In this paper, we focus on automatically finding existing fact-checks for claims made in social media posts (tweets). We conduct both classification and retrieval experiments, in monolingual (English only), multilingual (Spanish, Portuguese), and cross-lingual (Hindi-English) settings using multilingual transformer models such as XLM-RoBERTa and multilingual embeddings such as LaBSE and SBERT. We present promising results for "match" classification (93% average accuracy) in four language pairs. We also find that a BM25 baseline outperforms state-of-the-art multilingual embedding models for the retrieval task during our monolingual experiments. We highlight and discuss NLP challenges while addressing this problem in different languages, and we introduce a novel curated dataset of fact-checks and corresponding tweets for future research.

show abstract

Extractive and Abstractive Explanations for Fact-Checking and Evaluation of News

Cited by 8 publications

References 15 publications

The perils and promises of fact-checking with large language models

The perils and promises of fact-checking with large language models

Preventing profiling for ethical fake news detection

Matching Tweets With Applicable Fact-Checks Across Languages

Contact Info

Product

Resources

About