FacTeR-Check: Semi-automated fact-checking through Semantic Similarity and Natural Language Inference

Abstract

Our society produces and shares overwhelming amounts of information through the Online Social Networks (OSNs). Within this environment, misinformation and disinformation have proliferated, becoming a public safety concern on every country. Allowing the public and professionals to efficiently find reliable evidence about the factual veracity of a claim is crucial to mitigate this harmful spread. To this end, we propose FacTeR-Check, a multilingual architecture for semiautomated fact-checking that can be used for either the general public but also useful for factchecking organisations. FacTeR-Check enables retrieving fact-checked information, unchecked claims verification and tracking dangerous information over social media. This architectures involves several modules developed to evaluate semantic similarity, to calculate natural language inference and to retrieve information from Online Social Networks. The union of all these modules builds a semi-automated fact-checking tool able of verifying new claims, to extract related evidence, and to track the evolution of a hoax on a OSN. While individual modules are validated on related benchmarks (mainly MSTS and SICK), the complete architecture is validated using a new dataset called NLI19-SP that is publicly released with COVID-19 related hoaxes and tweets from Spanish social media. Our results show state-of-the-art performance on the individual benchmarks, as well as producing useful analysis of the evolution over time of 61 different hoaxes.

Who Are All The Stochastic Parrots Imitating? They Should Tell Us!

“…For that, two existing tasks can be used: 1) identifying which part of the generated text refers to the citation . 2) Validate that the citation is appropriate for the selected text span (Karadzhov et al, 2017;Martín et al, 2021;Honovich et al, 2022;Mihaylova et al, 2018;Lee et al, 2020). Using such automatic methods instead of manually verifying citations will result in faster model development.…”

Semantic similarity models for automated fact-checking: ClaimCheck as a claim matching tool

“…Specifically, the domain of Semantic Textual Similarity (STS) focuses on the task of measuring the similarity in meaning between sentences. In this regard, models assess the extent to which two claims are semantically, textually, lexically, and referentially similar (Hövelmeyer;Boland;Dietze, 2022;Martín et al, 2021;Sheng et al, 2021).…”

Desinformación y vacunas en redes

“…La recopilación, procesado y análisis de datos consta de cinco fases, inspirados en la metodología de FacTeR-Check (Martín et al, 2021): 1) se recolectan los bulos (en este caso de antivacunas) a partir de los desmentidos de fact-checkers; 2) se crean de forma automática las consultas (queries) que permiten realizar las búsquedas de tuits sobre los bulos considerados; 3) se utiliza la API (del inglés Application Programming Interface) de Twitter que facilita la descarga de los tuits mediante las consultas previamente definidas; 4) se emplea una técnica de Inteligencia Artificial conocida como Natural Language Inference (NLI) (MacCartney, 2009) para filtrar entre aquellas publicaciones que afirman o contradicen el bulo; 5) y, por último, se descargan los datos de los usuarios de tales tuits relacionados con el bulo.…”