Mitigation of Diachronic Bias in Fake News Detection Dataset

Abstract

Fake news causes significant damage to society. To deal with these fake news, several studies on building detection models and arranging datasets have been conducted. Most of the fake news datasets depend on a specific time period. Consequently, the detection models trained on such a dataset have difficulty detecting novel fake news generated by political changes and social changes; they may possibly result in biased output from the input, including specific person names and organizational names. We refer to this problem as Diachronic Bias because it is caused by the creation date of news in each dataset. In this study, we confirm the bias, especially proper nouns including person names, from the deviation of phrase appearances in each dataset. Based on these findings, we propose masking methods using Wikidata to mitigate the influence of person names and validate whether they make fake news detection models robust through experiments with in-domain and out-of-domain data.

A Review of Fake News Detection Techniques for Arabic Language

“…In 2018, a European Commission report defined "fake news" as "all forms of false, inaccurate, or misleading information designed, presented, and promoted to cause public harm intentionally or for profit" [17]. Research in [18] has provided and defined a set of terms related to fake news, including hoax, rumor, spam, misinformation, disinformation, clickbait, satire, propaganda, and hyperpartisan. Hoaxes are facts that are either erroneous or inaccurate and are presented as actual facts in news stories [19].…”

“…(-) May not fully convey the message"s context well as underlying intent, leading to misunderstandings [18].…”

Preventing profiling for ethical fake news detection

“…Regarding bias mitigation, several mitigation methods have been proposed ( Hovy & Prabhumoye, 2021 ). For example, Murayama, Wakamiya, and Aramaki (2021) explicitly masked person names in fake news detection models to tackle diachronic biases found in time-limited datasets. Binns, Veale, Van Kleek, and Shadbolt (2017) discussed normative bias and investigated how different norms about offensive speech among the data annotators can bias automated content moderation models.…”

The Battlefront of Combating Misinformation and Coping with Media Bias

“…We then cover recent approaches (Lee et al, 2021b; that leverage a combination of these elements for greater representation power and robustness. Importantly, we also cover works that explore the diachronic bias of fake news detection and portability across data in different time and language settings (Murayama et al, 2021;Gereme et al, 2021).…”