sthruggle at SemEval-2019 Task 5: An Ensemble Approach to Hate Speech Detection

Nourbakhsh, Aria; Vermeer, F.; Wiltvank, Gijs; Goot, Rob van der

doi:10.18653/v1/s19-2086

Cited by 8 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important step was the introduction of transformers, particularly BERT [21], which in a recent competition for hate speech detection provided seven out of the ten best performing models in a subtask [95]. It is also possible to use an ensemble of the above methods [59,61]. In fact, such an approach has recently provided the best performance (based on the average performance on all subtasks) in a competition among more than fifty participating teams [77].…”

Section: Related Workmentioning

confidence: 99%

Challenges of Hate Speech Detection in Social Media

2021

View full text Add to dashboard Cite

The detection of hate speech in social media is a crucial task. The uncontrolled spread of hate has the potential to gravely damage our society, and severely harm marginalized people or groups. A major arena for spreading hate speech online is social media. This significantly contributes to the difficulty of automatic detection, as social media posts include paralinguistic signals (e.g. emoticons, and hashtags), and their linguistic content contains plenty of poorly written text. Another difficulty is presented by the context-dependent nature of the task, and the lack of consensus on what constitutes as hate speech, which makes the task difficult even for humans. This makes the task of creating large labeled corpora difficult, and resource consuming. The problem posed by ungrammatical text has been largely mitigated by the recent emergence of deep neural network (DNN) architectures that have the capacity to efficiently learn various features. For this reason, we proposed a deep natural language processing (NLP) model—combining convolutional and recurrent layers—for the automatic detection of hate speech in social media data. We have applied our model on the HASOC2019 corpus, and attained a macro F1 score of 0.63 in hate speech detection on the test set of HASOC. The capacity of DNNs for efficient learning, however, also means an increased risk of overfitting. Particularly, with limited training data available (as was the case for HASOC). For this reason, we investigated different methods for expanding resources used. We have explored various opportunities, such as leveraging unlabeled data, similarly labeled corpora, as well as the use of novel models. Our results showed that by doing so, it was possible to significantly increase the classification score attained.

show abstract

Section: Related Workmentioning

confidence: 99%

Challenges of Hate Speech Detection in Social Media

2021

View full text Add to dashboard Cite

show abstract

“…What is more, transformer models have proved highly successful in hate speech detection competitions (with most of the top ten teams using a transformer in a recent challenge [46]). Ensembles of transformers also proved to be successful in hate speech detection [28,30]. So much so, that such a solution has attained the best performance (i.e.…”

Section: Related Workmentioning

confidence: 99%

Hate Speech Detection Using Transformer Ensembles on the HASOC Dataset

Alonso

Saini

Kovács

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

With the ubiquity and anonymity of the Internet, the spread of hate speech has been a growing concern for many years now. The language used for the purpose of dehumanizing, defaming or threatening individuals and marginalized groups not only threatens the mental health of its targets, as well as their democratic access to the Internet, but also the fabric of our society. Because of this, much effort has been devoted to manual moderation. The amount of data generated each day, particularly on social media platforms such as Facebook and twitter, however makes this a Sisyphean task. This has led to an increased demand for automatic methods of hate speech detection. Here, to contribute towards solving the task of hate speech detection, we worked with a simple ensemble of transformer models on a twitter-based hate speech benchmark. Using this method, we attained a weighted F1score of 0.8426, which we managed to further improve by leveraging more training data, achieving a weighted F1-score of 0.8504. Thus markedly outperforming the best performing system in the literature.

show abstract

“…The problem of competing definitions would result in a poor feature detection set that could not help identifying hate speech. The problem posed by ungrammatical text has mainly been used to mitigate the difficulty of automatically detecting hateful speech, particularly when users intentionally change keywords" spelling or avoid automatic content [7], [8].…”

Section: Introductionmentioning

confidence: 99%

A New Hate Speech Detection System based on Textual and Psychological Features

Alkomah¹,

Salati²,

Ma³

2022

IJACSA

View full text Add to dashboard Cite

Hate speech often spreads on social media and harms individuals and the community. Machine learning models have been proposed to detect hate speech in social media; however, several issues presently limit the performance of current approaches. One challenge is the issue of having diverse comprehensions of hate speech constructs which will lead to many speech categories and different interpretations. In addition, certain language-specific features, and short text issues, such as Twitter, exacerbate the problem. Moreover, current machine learning approaches lack universality due to small datasets and the adoption of a few features of hateful speech. This paper develops and builds new feature sets based on frequencies of textual tokens and psychological characteristics. Then, the study evaluates several machine learning methods over a large dataset. Results showed that the Random Forest and BERT methods are the most valuable for detecting hate speech content. Furthermore, the most dominant features that are helpful for hate speech detection methods combine psychological features and Term-Frequency Inverse Document-Frequency (TFIDF) features. Therefore, the proposed approach could identify hate speech on social media platforms like Twitter.

show abstract

sthruggle at SemEval-2019 Task 5: An Ensemble Approach to Hate Speech Detection

Cited by 8 publications

References 5 publications

Challenges of Hate Speech Detection in Social Media

Challenges of Hate Speech Detection in Social Media

Hate Speech Detection Using Transformer Ensembles on the HASOC Dataset

A New Hate Speech Detection System based on Textual and Psychological Features

Contact Info

Product

Resources

About