Latent Hatred: A Benchmark for Understanding Implicit Hate Speech

ElSherief, Mai; Ziems, Caleb; Muchlinski, David; Anupindi, Vaishnavi; Seybolt, Jordyn; Choudhury, Munmun De; Yang, Diyi

doi:10.48550/arxiv.2109.05322

Cited by 6 publications

(12 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hate speech takes different forms in different social platforms (Wiegand, Ruppenhofer, and Kleinbauer 2019) and across time (Florio et al 2020). It is often implicit (ElSherief et al 2021), targeting a variety of groups. Consequently, transfer learning remains a challenge for hate-speech detection, and annotated Parler data is needed in order to achieve accurate classification.…”

Section: Parler Data Annotationmentioning

confidence: 99%

Going Extreme: Comparative Analysis of Hate Speech in Parler and Gab

Israeli¹,

Tsur²

2022

Preprint

View full text Add to dashboard Cite

Social platforms such as Gab and Parler, branded as 'freespeech' networks, have seen a significant growth of their user base in recent years. This popularity is mainly attributed to the stricter moderation enforced by mainstream platforms such as Twitter, Facebook, and Reddit. In this work we provide the first large scale analysis of hate-speech on Parler. We experiment with an array of algorithms for hate-speech detection, demonstrating limitations of transfer learning in that domain, given the illusive and ever changing nature of the ways hate-speech is delivered. In order to improve classification accuracy we annotated 10K Parler posts, which we use to fine-tune a BERT classifier. Classification of individual posts is then leveraged for the classification of millions of users via label propagation over the social network. Classifying users by their propensity to disseminate hate, we find that hate mongers make 16.1% of Parler active users, and that they have distinct characteristics comparing to other user groups. We find that hate mongers are more active, more central and express distinct levels of sentiment and convey a distinct array of emotions like anger and sadness. We further complement our analysis by comparing the trends discovered in Parler and those found in Gab. To the best of our knowledge, this is among the first works to analyze hate speech in Parler in a quantitative manner and on the user level, and the first annotated dataset to be made available to the community.

show abstract

Section: Parler Data Annotationmentioning

confidence: 99%

Going Extreme: Comparative Analysis of Hate Speech in Parler and Gab

Israeli¹,

Tsur²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Most research focused on overt forms of hate speech, but explicit hate is more easily identifiable, e.g., by lexicon-based methods (Davidson et al, 2017). Recent research focused on implicitness of hate speech and proposed new datasets (Jurgens et al, 2019;ElSherief et al, 2021;Wiegand et al, 2021;Ocampo et al, 2023;Nejadgholi et al, 2022;Hartvigsen et al, 2022). A study developed a taxonomy of implicit hate and provided a labeled dataset (ElSherief et al, 2021), but its annotation scheme is not generalizable in the Korean context (e.g., White Grievance).…”

Section: Related Workmentioning

confidence: 99%

“…The value of 1 means there is room for disagreement about whether it is an offensive expression speech or not. This category includes implicit hate expressions, such as sarcasm, stereotypes of a target group, etc (ElSherief et al, 2021). The labeling decision could depend on the context of expression.…”

Section: A2 Detailed Guidelinementioning

confidence: 99%

“…(2) Implicitness: The majority of previous research conducted on this subject has primarily concentrated on explicit manifestations of offensive language. A few studies investigated the implicit form of hateful content (ElSherief et al, 2021;Ocampo et al, 2023;Hartvigsen et al, 2022), most of which targeted English. (3) Biased and noisy labels: Manual annotation is a predominant approach for constructing labeled data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Acceptance and Metamorphosis of New Culture at Soongsil School in Pyongyang during the Modern Transition Period - Introduction and Spread of Modern Western Music and Sports -

Park¹,

부교수²

2020

The Historical Association for Soong-Sil

View full text Add to dashboard Cite

Numerous datasets have been proposed to combat the spread of online hate. Despite these efforts, a majority of these resources are Englishcentric, primarily focusing on overt forms of hate. This research gap calls for developing high-quality corpora in diverse languages that also encapsulate more subtle hate expressions. This study introduces K-HATERS, a new corpus for hate speech detection in Korean, comprising approximately 192K news comments with target-specific offensiveness ratings. This resource is the largest offensive language corpus in Korean and is the first to offer targetspecific ratings on a three-point Likert scale, enabling the detection of hate expressions in Korean across varying degrees of offensiveness. We conduct experiments showing the effectiveness of the proposed corpus, including a comparison with existing datasets. Additionally, to address potential noise and bias in human annotations, we explore a novel idea of adopting the Cognitive Reflection Test, which is widely used in social science for assessing an individual's cognitive ability, as a proxy of labeling quality. Findings indicate that annotations from individuals with the lowest test scores tend to yield detection models that make biased predictions toward specific target groups and are less accurate. This study contributes to the NLP research on hate speech detection and resource construction. The code and dataset can be accessed at https://github. com/ssu-humane/K-HATERS.

show abstract

“…Recent works on hate speech understanding (Sap et al, 2019b;ElSherief et al, 2021;Huang et al, 2022) have considered training autoregressive language models to generate underlying explanations on hate speech. The models are trained on humanwritten free-text rationales such as implied statements and targeted groups.…”

Section: Introductionmentioning

confidence: 99%

HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning

Yang,

Kim,

Kim

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

With the proliferation of social media, accurate detection of hate speech has become critical to ensure safety online. To combat nuanced forms of hate speech, it is important to identify and thoroughly explain hate speech to help users understand its harmful effects. Recent benchmarks have attempted to tackle this issue by training generative models on free-text annotations of implications in hateful text. However, we find significant reasoning gaps in the existing annotations schemes, which may hinder the supervision of detection models. In this paper, we introduce a hate speech detection framework, HARE, which harnesses the reasoning capabilities of large language models (LLMs) to fill these gaps in explanations of hate speech, thus enabling effective supervision of detection models. Experiments on SBIC and Implicit Hate benchmarks show that our method, using model-generated data, consistently outperforms baselines, using existing free-text human annotations. Analysis demonstrates that our method enhances the explanation quality of trained models and improves generalization to unseen datasets. Our code is available at https://github.com/ joonkeekim/hare-hate-speech.git.

show abstract

Latent Hatred: A Benchmark for Understanding Implicit Hate Speech

Cited by 6 publications

References 46 publications

Going Extreme: Comparative Analysis of Hate Speech in Parler and Gab

Going Extreme: Comparative Analysis of Hate Speech in Parler and Gab

Acceptance and Metamorphosis of New Culture at Soongsil School in Pyongyang during the Modern Transition Period - Introduction and Spread of Modern Western Music and Sports -

HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning

Contact Info

Product

Resources

About