Dynamics of online hate and misinformation

Cinelli, Matteo; Pelicon, Andraž; Mozetič, Igor; Novak, Petra Kralj; Zollo, Fabiana

doi:10.1038/s41598-021-01487-w

Cited by 61 publications

(42 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, note that the level of agreement between the best model and the annotators is very close to the inter-annotator agreement. This result is comparable to other related datasets, where the annotation task is subjective and it is unrealistic to expect perfect agreement between the annotators [ 36 , 37 ]. If one accepts an inherent ambiguity of the hate speech classification task, there is very little room for improvement of the binary classification model.…”

Section: Resultssupporting

confidence: 83%

Retweet communities reveal the main sources of hate speech

et al. 2022

Self Cite

View full text Add to dashboard Cite

We address a challenging problem of identifying main sources of hate speech on Twitter. On one hand, we carefully annotate a large set of tweets for hate speech, and deploy advanced deep learning to produce high quality hate speech classification models. On the other hand, we create retweet networks, detect communities and monitor their evolution through time. This combined approach is applied to three years of Slovenian Twitter data. We report a number of interesting results. Hate speech is dominated by offensive tweets, related to political and ideological issues. The share of unacceptable tweets is moderately increasing with time, from the initial 20% to 30% by the end of 2020. Unacceptable tweets are retweeted significantly more often than acceptable tweets. About 60% of unacceptable tweets are produced by a single right-wing community of only moderate size. Institutional Twitter accounts and media accounts post significantly less unacceptable tweets than individual accounts. In fact, the main sources of unacceptable tweets are anonymous accounts, and accounts that were suspended or closed during the years 2018–2020.

show abstract

Section: Resultssupporting

confidence: 83%

Retweet communities reveal the main sources of hate speech

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“… 2016 ; Cinelli et al. 2021 ). However, the performance can be improved if user-related context is taken into account (Gao and Huang 2017 ; Fehn Unsvåg and Gambäck 2018 ).…”

Section: Discussionmentioning

confidence: 99%

Evolution of topics and hate speech in retweet network communities

et al. 2021

Self Cite

View full text Add to dashboard Cite

Twitter data exhibits several dimensions worth exploring: a network dimension in the form of links between the users, textual content of the tweets posted, and a temporal dimension as the time-stamped sequence of tweets and their retweets. In the paper, we combine analyses along all three dimensions: temporal evolution of retweet networks and communities, contents in terms of hate speech, and discussion topics. We apply the methods to a comprehensive set of all Slovenian tweets collected in the years 2018–2020. We find that politics and ideology are the prevailing topics despite the emergence of the Covid-19 pandemic. These two topics also attract the highest proportion of unacceptable tweets. Through time, the membership of retweet communities changes, but their topic distribution remains remarkably stable. Some retweet communities are strongly linked by external retweet influence and form super-communities. The super-community membership closely corresponds to the topic distribution: communities from the same super-community are very similar by the topic distribution, and communities from different super-communities are quite different in terms of discussion topics. However, we also find that even communities from the same super-community differ considerably in the proportion of unacceptable tweets they post.

show abstract

“…We used the Italian and Slovenian models in two previous analytical studies on hate speech in social media. The Italian model was used in a work investigating relationships between hate speech and misinformation sources on the Italian YouTube [4]. The Slovenian model was used to perform an analysis on the evolution of retweet communities, hate speech and topics on the Slovenian Twitter during 2018-2020 [8][9][10].…”

Section: Model Training and Evaluationmentioning

confidence: 99%

Handling Disagreement in Hate Speech Modelling

Novak

Scantamburlo

Pelicon

et al. 2022

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

Hate speech annotation for training machine learning models is an inherently ambiguous and subjective task. In this paper, we adopt a perspectivist approach to data annotation, model training and evaluation for hate speech classification. We first focus on the annotation process and argue that it drastically influences the final data quality. We then present three large hate speech datasets that incorporate annotator disagreement and use them to train and evaluate machine learning models. As the main point, we propose to evaluate machine learning models through the lens of disagreement by applying proper performance measures to evaluate both annotators’ agreement and models’ quality. We further argue that annotator agreement poses intrinsic limits to the performance achievable by models. When comparing models and annotators, we observed that they achieve consistent levels of agreement across datasets. We reflect upon our results and propose some methodological and ethical considerations that can stimulate the ongoing discussion on hate speech modelling and classification with disagreement.

show abstract

Dynamics of online hate and misinformation

Cited by 61 publications

References 43 publications

Retweet communities reveal the main sources of hate speech

Retweet communities reveal the main sources of hate speech

Evolution of topics and hate speech in retweet network communities

Handling Disagreement in Hate Speech Modelling

Contact Info

Product

Resources

About