Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets

Farooqi, Zaki Mustafa; Ghosh, Sreyan; Shah, Rajiv Ratn

doi:10.48550/arxiv.2112.09986

Cited by 4 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…About 7000 posts in code-mixed Hindi and English are classi ed as non-hate offensive (NOT) and hate and offensive (HOF). Few studies used this dataset to perform the subtask to classify the posts as NOT and HOF in code mixed language [27][28] [29]. Another dataset is there to detect hate speech and offensive content identi cation in Indo-European languages [15].…”

Section: Related Workmentioning

confidence: 99%

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

Sharma

Singh

Gupta

2023

Preprint

View full text Add to dashboard Cite

Social media has over the years provided a medium for creation and dissemination of opinions and thoughts through online platforms. While it allows users to express their views, sentiments and emotions, some people try to use it to generate and share unpleasant and hateful content. Such content is now referred to as hate speech and it may target an individual, a group, a community, or a country. During the last few years, several techniques have been developed to automatically detect and identify hate speech, offensive and abusive content from social media platforms. However, majority of the studies focused on hate speech detection in English language texts. With social media getting higher penetration across different geographies, there is now a significant amount of content generated in various languages. Though there have been significant advancements in algorithmic approaches for the task, the non-availability of suitable dataset in other languages poses a problem in research advancement in them. Hindi is one such widely spoken language where such datasets are not available. This work attempts to bridge this research gap by presenting a curated and annotated dataset for target-based hate speech (TABHATE) in the Hindi language. The dataset comprises of 2,020 tweets and is annotated by three independent annotators. A multiclass labelling is used where each tweet is labelled as: (i) individual targeting, (ii) community targeting, and (iii) none. Inter annotator agreement is computed. The suitability of dataset is then further explored by applying some standard deep learning and transformer-based models for the task of hate speech detection. The experimental results obtained show that the dataset can be used for experimental work on hate speech detection of Hindi language texts.

show abstract

Section: Related Workmentioning

confidence: 99%

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

Sharma

Singh

Gupta

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, multilingual transformers, particularly mBERT (multilingual BERT) or XLM-RoBERTa, have been implemented in the multilingual domain for hate speech detection tasks. These models have provided cutting-edge performance in crosslingual and multilingual settings, where several studies demonstrate their usefulness in many languages, especially in low-resource ones [19,20].…”

Section: Introductionmentioning

confidence: 99%

Multilingual Hate Speech Detection: A Semi-Supervised Generative Adversarial Approach

Mnassri,

Farahbakhsh,

Crespi

2024

Entropy

View full text Add to dashboard Cite

Social media platforms have surpassed cultural and linguistic boundaries, thus enabling online communication worldwide. However, the expanded use of various languages has intensified the challenge of online detection of hate speech content. Despite the release of multiple Natural Language Processing (NLP) solutions implementing cutting-edge machine learning techniques, the scarcity of data, especially labeled data, remains a considerable obstacle, which further requires the use of semisupervised approaches along with Generative Artificial Intelligence (Generative AI) techniques. This paper introduces an innovative approach, a multilingual semisupervised model combining Generative Adversarial Networks (GANs) and Pretrained Language Models (PLMs), more precisely mBERT and XLM-RoBERTa. Our approach proves its effectiveness in the detection of hate speech and offensive language in Indo-European languages (in English, German, and Hindi) when employing only 20% annotated data from the HASOC2019 dataset, thereby presenting significantly high performances in each of multilingual, zero-shot crosslingual, and monolingual training scenarios. Our study provides a robust mBERT-based semisupervised GAN model (SS-GAN-mBERT) that outperformed the XLM-RoBERTa-based model (SS-GAN-XLM) and reached an average F1 score boost of 9.23% and an accuracy increase of 5.75% over the baseline semisupervised mBERT model.

show abstract

“…The state-of-the-art hostile text detection methods are available for English language. To enhance the research in low resource Indian languages, we have studied various methods which can detect hostile posts in Hindi (1, 2, 6-8, 13, 14), Marathi (7,11,12), Bengali (12), Saudi (4), Roman Urdu (15,16), Tamil (17) and codemixed language (7,(18)(19)(20)(21)(22). Figure 1 shows basic approaches used for hostile post-detection.…”

Section: Introductionmentioning

confidence: 99%

Investigating Hostile Post Detection in Gujarati: A Machine Learning Approach

Rameshbhai,

Rana

2024

IJE

View full text Add to dashboard Cite

Hostile post on social media is a crucial issue for individuals, governments and organizations. There is a critical need for an automated system that can investigate and identify hostile posts from large-scale data. In India, Gujarati is the sixth most spoken language. In this work, we have constructed a major hostile post dataset in the Gujarati language. The data are collected from Twitter, Instagram and Facebook. Our dataset consists of 1,51,000 distinct comments having 10,000 manually annotated posts. These posts are labeled into the Hostile and Non-Hostile categories. We have used the dataset in two ways: (i) Original Gujarati Text Data and (ii) English data translated from Gujarati text. We have also checked the performance of pre-processing and without pre-processing data by removing extra symbols and substituting emoji descriptions in the text. We have conducted experiments using machine learning models based on supervised learning such as Support Vector Machine, Decision Tree, Random Forest, Gaussian Naive-Bayes, Logistic Regression, K-Nearest Neighbor and unsupervised learning based model such as k-means clustering. We have evaluated performance of these models for Bag-of-Words and TF-IDF feature extraction methods. It is observed that classification using TF-IDF features is efficient. Among these methods Logistic regression outperforms with an Accuracy of 0.68 and F1-score of 0.67. The purpose of this research is to create a benchmark dataset and provide baseline results for detecting hostile posts in Gujarati Language.

show abstract

Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets

Cited by 4 publications

References 37 publications

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

TABHATE: A Target-based Hate Speech Detection Dataset in Hindi

Multilingual Hate Speech Detection: A Semi-Supervised Generative Adversarial Approach

Investigating Hostile Post Detection in Gujarati: A Machine Learning Approach

Contact Info

Product

Resources

About