EdinburghNLP at WNUT-2020 Task 2: Leveraging Transformers with Generalized Augmentation for Identifying Informativeness in COVID-19 Tweets

Maveli, Nickil

doi:10.18653/v1/2020.wnut-1.67

Cited by 4 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This dataset was created for a shared task in WNUT-2020 on classifying tweets as "informative" or "uninformative", in providing information about "recovered, suspected, confirmed and death cases as well as location or travel history of the cases" [63]. Many of the models submitted in the competition were based on pre-trained transformer-based language models [3,12,33,33,53,61,61,67,87,91], including the first place model which was an ensemble of CT-BERT and RoBERTa [42], and the tied first place model which was a carefully tuned version of CT-BERT [58]. A 70-10-20 split was provided by the task organizers.…”

Section: 23mentioning

confidence: 99%

The Surprising Performance of Simple Baselines for Misinformation Detection

Pelrine

Danovitch

Rabbany

2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

As social media becomes increasingly prominent in our day to day lives, it is increasingly important to detect informative content and prevent the spread of disinformation and unverified rumours. While many sophisticated and successful models have been proposed in the literature, they are often compared with older NLP baselines such as SVMs, CNNs, and LSTMs. In this paper, we examine the performance of a broad set of modern transformer-based language models and show that with basic fine-tuning, these models are competitive with and can even significantly outperform recently proposed state-of-the-art methods. We present our framework as a baseline for creating and evaluating new methods for misinformation detection. We further study a comprehensive set of benchmark datasets, and discuss potential data leakage and the need for careful design of the experiments and understanding of datasets to account for confounding variables. As an extreme case example, we show that classifying only based on the first three digits of tweet ids, which contain information on the date, gives state-ofthe-art performance on a commonly used benchmark dataset for fake news detection -Twitter16. We provide a simple tool to detect this problem and suggest steps to mitigate it in future datasets. CCS CONCEPTS• Computing methodologies → Artificial intelligence; Natural language processing;

show abstract