Would Your Tweet Invoke Hate on the Fly? Forecasting Hate Intensity of Reply Threads on Twitter

Abstract

Curbing hate speech is undoubtedly a major challenge for online microblogging platforms like Twitter. While there have been studies around hate speech detection, it is not clear how hate speech finds its way into an online discussion. It is important for a content moderator to not only identify which tweet is hateful, but also to predict which tweet will be responsible for accumulating hate speech. This would help in prioritizing tweets that need constant monitoring. Our analysis reveals that for hate speech to manifest in an ongoing discussion, the source tweet may not necessarily be hateful; rather, there are plenty of such non-hateful tweets which gradually invoke hateful replies, resulting in the entire reply threads becoming provocative.In this paper, we define a novel problemgiven a source tweet and a few of its initial replies, the task is to forecast the hate intensity of upcoming replies. To this end, we curate a novel dataset constituting ∼ 4.5 contemporary tweets and their entire reply threads. Our preliminary analysis confirms that the evolution patterns along time of hate intensity among reply threads have highly diverse patterns, and there is no significant correlation between the hate intensity of the source tweets and that of their reply threads. We employ seven state-of-the-art dynamic models (either statistical signal processing or deep learning based) and show that they fail badly to forecast the hate intensity. We then propose DESSERT, a novel deep state-space model that leverages the function approximation capability of deep neural networks with the capacity to quantify the uncertainty of statistical signal processing models. Exhaustive experiments and ablation study show that DESSERT outperforms all the baselines substantially. Further, its deployment in an advanced AI platform designed to monitor real-world problematic hateful content has improved the aggregated insights extracted for countering the spread of online harms.T. Chakraborty would like to acknowledge the support of Logically, the Ramanujan Fellowship, and the Infosys Centre for AI, IIIT Delhi. We also thank Sarah Masud for her help in writing the paper. CCS CONCEPTS• Computing methodologies → Machine learning algorithms;• Information systems → Social tagging systems; • Humancentered computing → Social network analysis.

Predicting Hate Intensity of Twitter Conversation Threads

“…A few research looked into how hate speech spreads through social media. In a recent discussion, Dahiya et al [12] discussed the necessity of anticipating the hatred intensity of tweets. Lin et al [38] classified the tweets into different levels of hatred and suggested the HEAR model track posts that are likely to cause hate speech.…”

“…where 𝑐 𝜑 𝑖 refers to the 𝑖 𝑡ℎ reply. As the replies in the original dataset have no ground-truth hate intensity, we quantify the hate intensity of each reply using a weighted sum of two measures as suggested in [12]:…”

“…To avoid noise and drastic fluctuations that are not in line with reality, we further smooth the hate intensity sequence by utilizing a rolling average operation with window size 𝛿 [12]. A window consists of 𝛿 consecutive replies, and the window that starts from the 𝑘th reply is denoted by T 𝜑 𝑘,𝑘+𝛿 .…”

Battling Hateful Content in Indic Languages HASOC '21

“…Owing to the conversational design of social media wherein users can reply to a given comment (either support, refute or irrelevant to the original message), the build-up of threads in response to a hateful message can also intensify hate even if the reply is not hateful on its own. The evolution of such hate intensity has shown diverse patterns and no direct correlation to the parent tweet which makes the task of hate speech detection more difficult [10].…”

“…Due to lack of resources for Marathi we catalogue 9 a list of profane words in Marathi and use this to find the fraction of profane words in a tweet. For Hindi, we curate a list of profane words by collating and appending to existing lists 10 , and use this to score each tweet. As for the sentiment of the tweet, we incorporated off-theshelf HuggingFace models to obtain the positive, negative and neutral scores for a tweet 11 12 .…”