SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research

Social media offers the potential to provide detection of outbreaks or public health incidents faster than traditional reporting mechanisms. In this paper, we developed and tested a pipeline to produce alerts of influenza-like illness (ILI) using Twitter data. Data was collected from the Twitter API, querying keywords referring to ILI symptoms and geolocated to Wales. Tweets that described first-hand descriptions of symptoms (as opposed to non-personal descriptions) were classified using transformer-based language models specialised on social media (BERTweet and TimeLMs), which were trained on a manually labelled dataset matching the above criteria. After gathering this data, weekly tweet counts were applied to the regression-based Noufaily algorithm to identify exceedances throughout 2022. The algorithm was also applied to counts of ILI-related GP consultations for comparison. Exceedance detection applied to the classified tweet counts produced alerts starting four weeks earlier than by using GP consultation data. These results demonstrate the potential to facilitate advanced preparedness for unexpected increases in healthcare burdens.

show abstract

Reciprocal Communication and Political Deliberation on Twitter

Ackland,

Gumbert,

Pütz

et al. 2023

Social Sciences

View full text Add to dashboard Cite

Social media platforms such as Twitter/X are increasingly important for political communication but the empirical question as to whether such communication enhances democratic consensus building (the ideal of deliberative democracy) or instead contributes to societal polarisation via fostering of hate speech and “information disorders” such as echo chambers is worth exploring. Political deliberation involves reciprocal communication between users, but much of the recent research into politics on social media has focused on one-to-many communication, in particular the sharing and diffusion of information on Twitter via retweets. This paper presents a new approach to studying reciprocal political communication on Twitter, with a focus on extending network-analytic indicators of deliberation. We use the Twitter v2 API to collect a new dataset (#debatenight2020) of reciprocal communication on Twitter during the first debate of the 2020 US presidential election and show that a hashtag-based collection alone would have collected only 1% of the debate-related communication. Previous work into using social network analysis to measure deliberation has involved using discussion tree networks to quantify the extent of argumentation (maximum depth) and representation (maximum width); we extend these measures by explicitly incorporating reciprocal communication (via triad census) and the political partisanship of users (inferred via usage of partisan hashtags). Using these methods, we find evidence for reciprocal communication among partisan actors, but also point to a need for further research to understand what forms this communication takes.

show abstract

SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research

Cited by 3 publications

References 56 publications

UFEL: a By-Design Understandable and Frugal Entity Linking System for French Microposts

UFEL: a By-Design Understandable and Frugal Entity Linking System for French Microposts

Towards a Social Media-based Disease Surveillance System for Early Detection of Influenza-like Illnesses: A Twitter Case Study inWales

Reciprocal Communication and Political Deliberation on Twitter

Contact Info

Product

Resources

About