Automated Detection of Cyberbullying Against Women and Immigrants and Cross-domain Adaptability

Abstract

Cyberbullying is a prevalent and growing social problem due to the surge of social media technology usage. Minorities, women, and adolescents are among the common victims of cyberbullying. Despite the advancement of NLP technologies, the automated cyberbullying detection remains challenging. This paper focuses on advancing the technology using state-of-the-art NLP techniques. We use a Twitter dataset from SemEval 2019 -Task 5 (HatEval) on hate speech against women and immigrants. Our best performing ensemble model based on DistilBERT has achieved 0.73 and 0.74 of F1 score in the task of classifying hate speech (Task A) and aggressiveness and target (Task B) respectively. We adapt the ensemble model developed for Task A to classify offensive language in external datasets and achieved ∼0.7 of F1 score using three benchmark datasets, enabling promising results for cross-domain adaptability. We conduct a qualitative analysis of misclassified tweets to provide insightful recommendations for future cyberbullying research.

Cyberbullying Detection using Ensemble Method

“…Herath et al [63] developed and evaluated a cyberbullying classification model using DistilBERT and state-of-the-art NLP technology. The dataset collected from…”

“…Three models which were built on a Training dataset by changing the ratios of the majority classes acts as base models. The final ensemble model was built using a Simple Voting Classifier [63]. Since the dataset used in this thesis is comparatively balanced, a simple DistilBERT model was developed and the results are discussed below:…”

“…Ask.fm dataset was utilized to categorize the participant roles into victim and harasser, which is a multi-class classification problem. The evaluation of cyberbullying classification was done based on the model developed by Herath et al[63] as explained above, where in three models where ensembled in which each model was fed with a training dataset in which the majority class was positive, negative and balanced. The Twitter dataset was used to evaluate this model, in which the tweets were categorized into "Offensive" and "Not Offensive".…”

Cyberbullying Detection using Ensemble Method

“…The TF-IDF algorithm is based on word statistics for text feature extraction. TF-IDF is used to vectorize the input [1]. The model considers only the expression of words, that are similar in all texts.…”

“…This has resulted in bullying general or specific users and user groups, either knowingly or unknowingly. The abuses resulting from cyberbullying can cause psychological harm to the target users and groups [1].…”