Model Generalization on COVID-19 Fake News Detection

Abstract

Amid the pandemic COVID-19, the world is facing unprecedented infodemic with the proliferation of both fake and real information. Considering the problematic consequences that the COVID-19 fake-news have brought, the scientific community has put effort to tackle it. To contribute to this fight against the infodemic, we aim to achieve a robust model for the COVID-19 fake-news detection task proposed at CONSTRAINT 2021 (FakeNews-19) by taking two separate approaches: 1) fine-tuning transformers based language models with robust loss functions and 2) removing harmful training instances through influence calculation. We further evaluate the robustness of our models by evaluating on different COVID-19 misinformation test set (Tweets-19) to understand model generalization ability. With the first approach, we achieve 98.13% for weighted F1 score (W-F1) for the shared task, whereas 38.18% W-F1 on the Tweets-19 highest. On the contrary, by performing influence data cleansing, our model with 99% cleansing percentage can achieve 54.33% W-F1 score on Tweets-19 with a trade-off. By evaluating our models on two COVID-19 fake-news test sets, we suggest the importance of model generalization ability in this task to step forward to tackle the COVID-19 fake-news problem in online social media platforms.

A Holistic Framework for Analyzing the COVID-19 Vaccine Debate

“…Recent studies have noted the prevalence of rumors and misinformation in the context of the Covid-19 pandemic (Loomba et al, 2021;Shahi et al, 2021;Lazarus et al, 2021;Ahmed et al, 2020). Following this trend, several computational approaches have been proposed to detect misinformation related to Covid in news outlets and social media Bang et al, 2021;Serrano et al, 2020;Al-Rakhami and Al-Amri, 2020). In this paper, we take a different approach and look at the problem of identifying opinions surrounding the Covid-19 vaccine, and explicitly modeling the rationale and moral sentiment that motivates them.…”

A Holistic Framework for Analyzing the COVID-19 Vaccine Debate

“…Recent studies have noted the prevalence of rumors and misinformation in the context of the Covid-19 pandemic (Loomba et al, 2021;Shahi et al, 2021;Lazarus et al, 2021;Ahmed et al, 2020). Following this trend, several computational approaches have been proposed to detect misinformation related to Covid in news outlets and social media Bang et al, 2021;Serrano et al, 2020 Amri, 2020). In this paper, we take a different approach and look at the problem of identifying opinions surrounding the Covid-19 vaccine, and explicitly modeling the rationale and moral sentiment that motivates them.…”

Sentiment Analysis and Topic Modeling for COVID-19 Vaccine Discussions

“…Furthermore, an Infodemic Risk Index was developed to capture the magnitude of exposure to unreliable news across countries. To contribute to the fight against the infodemic, Bang et al [21] aimed to achieve a robust model for the COVID-19 fake-news detection task proposed in CONSTRAINT 2021 (FakeNews-19). They further improved the robustness of the model by evaluating different COVID-19 misinformation test sets (Tweets-19) to further improve the gen-eralization ability of the model to solve the COVID-19 fake news problem in online social media platforms.…”