A domain-adaptive pre-training approach for language bias detection in news

Abstract

Media bias is a multi-faceted construct influencing individual behavior and collective decision-making. Slanted news reporting is the result of one-sided and polarized writing which can occur in various forms. In this work, we focus on an important form of media bias, i.e. bias by word choice. Detecting biased word choices is a challenging task due to its linguistic complexity and the lack of representative gold-standard corpora. We present DA-RoBERTa, a new state-of-the-art transformer-based model adapted to the media bias domain which identifies sentence-level bias with an F1 score of 0.814. In addition, we also train, DA-BERT and DA-BART, two more transformer models adapted to the bias domain. Our proposed domain-adapted models outperform prior bias detection approaches on the same data. CCS CONCEPTS• Computing methodologies → Natural language processing; • Information systems → Clustering and classification.

Specialized document embeddings for aspect-based similarity of research papers

“…Reimers and Gurevych [60] show representations from BERT [14] can be improved with a Siamese architecture [7] when fine-tuned on semantic textual similarity datasets. Other approaches augment pre-trained models (e.g., BART [39], RoBERTa [42]) combining separate trained intermediate tasks and external knowledge sources to solve an additional final task, such as word sense disambiguation [73], paraphrase detection [71,72], fake news detection [70], and media bias detection [35,66]. Also, Cohan et al [12] use citations as a pretraining objective for a scientific BERT language model.…”