Homophobic and Hate Speech Detection Using Multilingual-BERT Model on Turkish Social Media

Abstract

Homophobic expressions are a form of insulting the sexual orientation or personality of people. Severe psychological traumas may occur in people who are exposed to this type of communication. It is important to develop automatic classification systems based on language models to examine social media content and distinguish homophobic discourse. This study aims to present a pre-trained Multilingual Bidirectional Encoder Representations from Transformers (M-BERT) model that can successfully detect whether Turkish comments on social media contain homophobic or related hate comments (i.e., sexist, severe humiliation, and defecation expressions). Comments in the Homophobic-Abusive Turkish Comments (HATC) dataset were collected from Instagram to train the detection models. The HATC dataset was manually labeled at the sentence level and combined with the Abusive Turkish Comments (ATC) dataset that has developed in our previous study. The HATC dataset has been balanced using the resampling method and two forms of the dataset (i.e., resHATC and original HATC) were used in the experiments. Afterward, the M-BERT model was compared with DL-based models (i.e., Long-Short Term Memory, Bidirectional Long-Short Term Memory (BiLSTM), Gated Recurrent Unit), Traditional Machine Learning (TML) classifiers (i.e., Support Vector Machine, Naive Bayes, Random Forest) and Ensemble Classifiers (i.e., Adaptive Boosting, eXtreme Gradient Boosting, Gradient Boosting) for the best model selection. The performance of the detection models was evaluated using F1-score, precision, and recall performance metrics. Results showed the best performance (homophobic F1-score: 82.64%, hateful F1-score: 91.75%, neutral F1-score: 96.08%, average F1-score: 90.15%) was achieved with the M-BERT model on the HATC dataset. The M-BERT detection model can increase the effectiveness of filters in detecting Turkish homophobic and related hate speech in social networks. It can be used to detect homophobic and related hate speech for different languages since the M-BERT model has multilingual pre-trained data.

Detecting homophobic and transphobic texts from youtube comments using machine learning models

“…The authors of [7] proposed multiple languages Deep Bidirectional Portrayals from Transformers (M-BERT) model that can effectively inform if Turkish comments on social media comprise homophobic or comparable hateful speech. The Homophobic-Abusive Turkish Comments (HATC) set of data was composed of remarks from Instagram that were utilized to prepare the detection systems.…”

Dealing imbalance dataset problem in sentiment analysis of recession in Indonesia

“…Therefore, appropriate analysis techniques are needed to analyze many texts relatively quickly [13]. Sentiment analysis is a technique that is executed automatically to obtain personal information to understand sentiment from text data sources [14]. Sentiment analysis is a technique to extract text data to obtain information about positive, neutral, or negative sentiments [15].…”

Combating Hate: How Multilingual Transformers Can Help Detect Topical Hate Speech

“…Karayiğit, Akdagli, & Aci (2022) investigated binary and multi-class classification on a Turkish dataset. They collected posts that contained homophobic remarks (Karayiğit, Akdagli, & Aci, 2022). The authors applied over and under-sampling to correct class imbalances and evaluated various architectures, including ensemble learning.…”

“…The fine-tuned mBERT model produced an F1 score of 0.90. The authors proposed that the model was performant was due to the size of the Turkish corpus that mBERT was originally trained on (Karayiğit, Akdagli, & Aci, 2022).…”

“…However, these experiments were performed on English datasets. Studies have also used target-specific datasets where samples were collected for the assertations made on a target group (Basile, et al, 2019;Karayiğit, Akdagli, & Aci, 2022;Ishman, 2020). We believe that topically focused hate speech detection should solve multi-class outcomes.…”