Tamil Offensive Language Detection: Supervised versus Unsupervised Learning Approaches

Abstract

Studies on Natural Language Processing are mainly conducted in English, with very few exploring languages that are under-resourced, including the Dravidian languages. We present a novel work in detecting offensive language using a corpus collected from YouTube containing comments in Tamil. The study specifically aims to compare two machine learning approaches, namely, supervised, and unsupervised to detect offensive patterns in textual communications. In the first setup, offensive language detection models were developed using the traditional machine learning algorithms such as Random Forest, Logistic Regression, Support Vector Machine and AdaBoost, and assessed based on human labeling. Conversely, we used K-means (K = 2) to cluster the unlabeled data before training the same set of machine learning algorithms to detect offensive communications. Performance scores indicate unsupervised clustering to be more effective than human labeling with ensemble classifiers achieving an impressive accuracy of 99.70% and 99.87%, respectively for balanced and imbalanced datasets, hence showing that unsupervised approach can be used effectively to detect offensive language in low resourced languages.

Offensive Language Detection on Social Media using Machine Learning

“…Furthermore, the ethical implications of deploying automated systems for hate speech detection cannot be overlooked. Concerns about privacy, freedom of speech, and the potential for over-surveillance are paramount [29]. The risk of false positives-where benign content is mistakenly classified as hate speech-poses a threat to free expression and could result in unwarranted censorship [30].…”

Taming Toxic Talk: Detecting Offensive Online Content in Hausa

“…A feature shared by the above studies is the focus on addressing cyberbullying challenges associated with high-resource languages. There is an increasing interest in the detection of offensive content and hate speech in low-resource languages such as Tamil [34,35], Pashto [36], Urdu [37], Persian [38]. Similar studies have been focused on improving resources for tackling offensive and hateful content detection [39,40,41,42].…”