Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection

Abstract

We present the results and the main findings of the NLP4IF-2021 shared tasks. Task 1 focused on fighting the COVID-19 infodemic in social media, and it was offered in Arabic, Bulgarian, and English. Given a tweet, it asked to predict whether that tweet contains a verifiable claim, and if so, whether it is likely to be false, is of general interest, is likely to be harmful, and is worthy of manual fact-checking; also, whether it is harmful to society, and whether it requires the attention of policy makers. Task 2 focused on censorship detection, and was offered in Chinese. A total of ten teams submitted systems for task 1, and one team participated in task 2; nine teams also submitted a system description paper. Here, we present the tasks, analyze the results, and discuss the system submissions and the methods they used. Most submissions achieved sizable improvements over several baselines, and the best systems used pre-trained Transformers and ensembles. The data, the scorers and the leaderboards for the tasks are available at http:// gitlab.com/NLP4IF/nlp4if-2021.

Toxic language detection: A systematic review of Arabic datasets

“…Dataset repository (Ousidhoum et al, 2019) (Alshalan & Al-Khalifa, 2020) (Chowdhury et al, 2020) (Zampieri et al, 2020) (Röttger et al, 2022) Annotation platform All those works published their guidelines along with the project page in the annotation platform: https://micromappers.qcri.org/, and they are accessible by everyone. See for example the annotation guidelines of the different versions of the COVID-19 dataset Nakov et al, 2022;Shaar et al, 2021), published in each project page in: https://micromappers.qcri.org/project/category/covid-19/.…”

Online to Offline Crossover of White Supremacist Propaganda

“…In recent years, detection of computational propaganda has made significant progress. Several shared tasks involving detection and classification of annotated propaganda datasets have been constructed for the development of detection and analysis techniques within the community [38,50]. Detection methods involve constructing machine learning models for identification of the use of propaganda techniques [16] and the classification of the types of techniques employed [6] within texts, and the use of network analysis approaches to identify propaganda through the presence of coordinated inauthentic action [15].…”

The CLEF-2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection

“…There has been a lot of research on checking the factuality of a claim, of a news article, or of an information source [6,8,41,45,51,59,64,68,86]. Special attention has been paid to disinformation and misinformation in social media [30,43,49,74,78,84], more recently with focus on fighting the COVID-19 infodemic [2,3,52,53].…”

“…is related to several tasks at SemEval: on determining rumour veracity [22,29], on stance detection [50], on fact-checking in community question answering forums [48], on propaganda detection [21,23], and on semantic textual similarity [1,58]. It is also related to the FEVER task [79] on fact extraction and verification, to the Fake News Challenge [31,63], to the FakeNews task at MediaEval [62], as well as to the NLP4IF tasks on propaganda detection [20] and on fighting the COVID-19 infodemic in social media [68].…”