Automated Hate Speech Detection and the Problem of Offensive Language

Davidson, Thomas; Warmsley, Dana; Macy, Michael W.; Weber, Ingmar

doi:10.1609/icwsm.v11i1.14955

Cited by 1,435 publications

(533 citation statements)

References 9 publications

Supporting

Mentioning

338

Contrasting

Unclassified

Order By: Relevance

“…The Davidson corpus (Davidson et al, 2017) is a tweet corpus annotated in terms of hate speech, offensive speech or neither. The corpus contains 24,802 tweets: 76% are offensive, 11.4% are hateful, and 16.6% are neither.…”

Section: Methodsmentioning

confidence: 99%

“…Waseem and Hovy (2016) employed character-level features with logistic regression to classify tweets. Davidson et al (2017) classified tweets using word-level features, partof-speech, sentiment and meta-data of tweets with a logistic regression classifier. Other hard-coded features have been used for hate speech detection, such as user features (Fehn Unsvåg and Gambäck, 2018).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multiword Expression Features for Automatic Hate Speech Detection

Zampieri

Illina

Fohr

2021

Natural Language Processing and Information Systems

View full text Add to dashboard Cite

The task of automatically detecting hate speech in social media is gaining more and more attention. Given the enormous volume of content posted daily, human monitoring of hate speech is unfeasible. In this work, we propose new word-level features for automatic hate speech detection (HSD): multiword expressions (MWEs). MWEs are lexical units greater than a word that have idiomatic and compositional meanings. We propose to integrate MWE features in a deep neural network-based HSD framework. Our baseline HSD system relies on Universal Sentence Encoder (USE). To incorporate MWE features, we create a three-branch deep neural network: one branch for USE, one for MWE categories, and one for MWE embeddings. We conduct experiments on two hate speech tweet corpora with different MWE categories and with two types of MWE embeddings, word2vec and BERT. Our experiments demonstrate that the proposed HSD system with MWE features significantly outperforms the baseline system in terms of macro-F1.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Multiword Expression Features for Automatic Hate Speech Detection

Zampieri

Illina

Fohr

2021

Natural Language Processing and Information Systems

View full text Add to dashboard Cite

show abstract

“…Most of hate speech and offensive language corpora are proposed for the English language [2,15,[44][45][46][47]. For the French language, a corpus of Facebook and Twitter annotated data for Islamophobia, sexism, homophobia, religion intolerance and disability detection was also proposed [48,49].…”

Section: Word Generalizationmentioning

confidence: 99%

Contextual-Aware and Expert Resources for Brazilian Portuguese Hate Speech Detection

Vargas

Carvalho

Góes

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper provides the first large-scale expert annotated corpus of Brazilian Instagram comments, a multilayer annotation schema for hate speech and offensive language detection on social media, and a contextual-aware offensive lexicon annotated with contextual information. The HateBR corpus was collected from the comment section of Brazilian politicians' accounts on Instagram and manually annotated by specialists. It is composed of 7,000 documents annotated according to three different layers: a binary classification (offensive versus non-offensive comments), offensiveness-level classification (highly, moderately, and slightly), and nine hate speech groups (xenophobia, racism, homophobia, sexism, religious intolerance, partyism, apology for the dictatorship, antisemitism, and fatphobia). The proposed specialized lexicon was manually identified by a linguist from the proposed corpus, which holds 1,000 explicit and implicit pejorative terms and expressions annotated with contextual information. Both the corpus and the contextual-aware offensive lexicon were annotated by three different experts and achieved high inter-annotator agreement. Lastly, we implemented baseline experiments on our corpus and lexicon. The obtained results outperform the current state-of-the-art for the Portuguese language, and the models which embody our specialized lexicon also present relevant performance in different languages. Accordingly, we hope that the proposed resources foster research on hate speech and offensive language detection in the Natural Language Processing area.

show abstract

“…The data sets used in this research are grouped into multiclass ( [6]) and binary classifications ( [12]).…”

Section: Comparative Analysismentioning

confidence: 99%

A Framework for Optimizing Ensemble Learning for Offensive Tweet Identification

Mullah

Zainon

Wahab

2022

Preprint

View full text Add to dashboard Cite

Online derogatory comments are ubiquitous on social media and areraising serious concerns across the globe. Social media data is riddenwith high dimensional search space due to noise, redundant features,and non-standardized writing style. These problems lead to high computationalcosts, longer training time, and low predictive accuracy inmachine learning models. The researchers proposed a framework for optimizingstacked generalized ensemble learning to address these problemsand enhance model performance. The main components of the frameworkinclude feature optimizer (FO), ensemble classifiers, and stratifiedK-foldCV (skfCV) through stacked generalization ensemble architecture.The ensemble classifiers, FO, and skfCV components make our methodstable and computationally efficient with the best performance. Theproposed method was validated using three benchmark datasets. The proposed method outperformed the state-of-the-art results in all theevaluation metrics used in those three articles adopted for comparison.

show abstract

Automated Hate Speech Detection and the Problem of Offensive Language

Cited by 1,435 publications

References 9 publications

Multiword Expression Features for Automatic Hate Speech Detection

Multiword Expression Features for Automatic Hate Speech Detection

Contextual-Aware and Expert Resources for Brazilian Portuguese Hate Speech Detection

A Framework for Optimizing Ensemble Learning for Offensive Tweet Identification

Contact Info

Product

Resources

About