Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims

Abstract

5469task.According to the seminal work by Shaar et al. (2020), the task is tackled by a two-stage information retrieval approach. Its typical workflow is illustrated in Figure 1(a). Given a claim as a query, in the first stage a basic searcher (e.g., BM25 Robertson and Zaragoza, 2009) searches for candidate articles from a collection of fact-checking articles (FC-articles). In the second stage, a more powerful model (e.g., BERT, Devlin et al., 2019) reranks the candidates to provide evidence for manual or automatic detection. Existing works focus on the reranking stage: Vo and Lee (2020) model the interactions between a claim and the whole candidate articles, while Shaar et al. ( 2020) extract several semantically similar sentences from FC-articles as a proxy. Nevertheless, these methods treat FCarticles as general documents and ignore characteristics of FC-articles. Figure 1(b) shows three sentences from candidate articles for the given claim. Among them, S1 is more friendly to semantic matching than S2 and S3 because the whole S1 focuses on describing its topic and does not contain tokens irrelevant to the given claim, e.g., "has spread over years" in S2. Thus, a semantic-based model does not require to have strong filtering capability. If we use only general methods on this task, the relevant S2 and S3 may be neglected while irrelevant S1 is focused. To let the model focus on key sentences (i.e., sentences as a good proxy of article-level relevance) like S2 and S3, we need to consider two characteristics of FC-articles besides semantics: C1. Claims are often quoted to describe the checked events (e.g., the underlined text in S2); C2. Event-irrelevant patterns to introduce or debunk claims are common in FC-articles (e.g., bold texts in S2 and S3).

That is a Known Lie: Detecting Previously Fact-Checked Claims

“…Another recent work by Sheng et al (2021) highlighted the importance of using lexical, semantic, and pattern-based information and proposed a reranker based on memory-enhanced transformers for claim matching.…”

Zoom Out and Observe: News Environment Perception for Fake News Detection

“…Chinese Dataset Post: We merged the non-overlapping parts of multiple Weibo datasets from (Ma et al, 2016) (excluding those unverified), (Song et al, 2019), (Zhang et al, 2021) and (Sheng et al, 2021a) to (Shaar et al, 2020). For posts or claims from fact-checking websites, we used the provided claim dates instead of the publication dates of the fact-checking articles, to avoid potential data contamination where the later news environment is more likely to contain corresponding fact-checking news and support direct fact verification.…”

“…9 bert-base-chinese and bert-base-uncased 10 We attempted to collect webpages using our posts as queries as Popat et al (2018) did but rare ones could serve as evidence except fact-checking articles. As an alternative, we directly used articles from (Sheng et al, 2021a) for Chinese and collected ~8k articles from a well-known fact-checking website Snopes.com for English.…”

Characterizing Multi-Domain False News and Underlying User Effects on Chinese Weibo

“…Comment-based methods employed crowd wisdom for prediction (Ruchansky, Seo and Liu, 2017;Shu et al, 2019a;Tian, Liu, Yang, Lyu, Zhang and Fang, 2020). For those that have been previously fact-checked claims, debunking articles are used for matching with news posts (Shaar, Babulkov, Da San Martino and Nakov, 2020;Vo and Lee, 2020;Sheng, Cao, Zhang, Li and Zhong, 2021a;Mansour, Elsayed and Al-Ali, 2022). The scope of evidence is broadened to evidential web articles (Popat, Mukherjee, Yates and Weikum, 2018b;Wu, Rao, Yang, Wang and Nazir, 2020;Wu, Rao, Zhang, Zhao and Nazir, 2021) or contemporary mainstream news (Sheng, Cao, Zhang, Li, Wang and Zhu, 2022).…”