HOTTEST: Hate and Offensive content identification in Tamil using Transformers and Enhanced STemming

Rajalakshmi, Ratnavel; Selvaraj, Srivarshan; R, Faerie Mattins; Vasudevan, Pavitra; M, Anand Kumar

doi:10.1016/j.csl.2022.101464

Cited by 17 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A feature shared by the above studies is the focus on addressing cyberbullying challenges associated with high-resource languages. There is an increasing interest in the detection of offensive content and hate speech in low-resource languages such as Tamil [34,35], Pashto [36], Urdu [37], Persian [38]. Similar studies have been focused on improving resources for tackling offensive and hateful content detection [39,40,41,42].…”

Section: Offensive Content Detectionmentioning

confidence: 99%

Taming Toxic Talk: Detecting Offensive Online Content in Hausa

Adam,

Zandam,

Inuwa-Dutse

2024

Preprint

View full text Add to dashboard Cite

Hausa, a major Chadic language spoken by over 100 million people in Africa, faces a challenge in the digital age. While widely used, it is considered a low-resource language from a computational linguistic perspective. This means there are limited resources and tools to analyse Hausa text, making it difficult to detect offensive and threatening language online. Our study aimed to bridge this gap. We conducted two user studies ($n=180$) to understand cyberbullying in Hausa. We then created the first-ever dataset of offensive and threatening Hausa phrases to train detection systems. We developed a system to flag such content and compared it to Google translation's ability to detect these terms. Our findings revealed a concerning trend: offensive and threatening language is prevalent online, especially in discussions about religion and politics. Our detection system was able to detect more than 70\% of offensive and threatening content, although many of these were mistranslated by Google's translation engine. We attribute this to the subtle relationship between offensive and threatening content and idiomatic expressions in the Hausa language. This highlights the importance of considering cultural nuances and idiomatic expressions in Hausa. To create a safer online environment for Hausa speakers, we recommend involving diverse stakeholders who understand local contexts and demographics. This will allow for the development of more accurate detection systems and targeted moderation strategies. Trigger Warning: Readers may find some of the terms in this study distressing or disturbing; all examples are for illustration only.

show abstract

Section: Offensive Content Detectionmentioning

confidence: 99%

Taming Toxic Talk: Detecting Offensive Online Content in Hausa

Adam,

Zandam,

Inuwa-Dutse

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The results showed that Stacking-LR provided the highest ACC of the three cases. In a recent study [29], the authors combine linguistic-based analysis with ensemble learning techniques to analyze Tamil text found in YouTube comments. The text representations were generated by employing a combination of data stemming and the utilization of the MuRIL pre-trained transformer developed by Google.…”

Section: B Ensemble Technique In Sentimental Analysismentioning

confidence: 99%

SETAR: Stacking Ensemble Learning for Thai Sentiment Analysis Using RoBERTa and Hybrid Feature Representation

Thiengburanathum,

Charoenkwan

2023

IEEE Access

View full text Add to dashboard Cite

Sentiment classification of social media posts is among the most challenging and timeconsuming tasks for analysts. This is particularly true when applied to languages that employ scriptio continua, such as the Thai language, in which there are no spaces between written words and where there is no end of sentence punctuation. Thai is considered a scarce-resource language as few datasets are available to researchers. Although machine-learning (ML) and deep-learning (DL) algorithms can identify sentiment classification polarity, the performance of the existing classification models are still inadequate. This study proposes a novel stacking ensemble learning technique for identifying sentiment classification polarity in the Thai language, SETAR. Our stacking ensemble strategy utilized the pre-trained Thai language model (WangChanBERTa), based on a Robustly Optimized BERT Pretraining Approach (RoBERTa) architecture to form a feature vector. This feature was combined with three distinct feature vectors obtained from three well-known categories, namely Word2Vec, TF-IDF, and bag-of-words, as a new hybrid sentence representation. The base learners were trained using seven chosen complex heterogeneous ML algorithms, including support vector machine (SVM), random forest (RF), extremely randomized trees (ET), light gradient boosting machine (LGBM), multi-layer perceptron (MLP), partial least squares (PLS), and logistic regression (LR) to enable the development of the final meta-learners. The results revealed that our proposed stacking ensemble model outperformed the baseline models of all classification metrics among the training and test sets, as was determined by extensive benchmarking, carried out on the four datasets, which included our developed sentiment corpus that domain experts annotated.

show abstract

“…The RoBERTa model has exhibited state-of-the-art performance for crosslingual and multi-lingual NLP tasks [12]. Likewise, the MuRIL model demonstrated benchmark performance in mono-lingual and multi-lingual classification tasks for Indian languages [13].…”

Section: Introductionmentioning

confidence: 99%

Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text Identification

Rehan,

Malik,

Jamjoom

2023

IEEE Access

View full text Add to dashboard Cite

Threatening content detection on social media has recently gained attention. There is very limited work regarding threatening content detection in low-resource languages, especially in Urdu. Furthermore, previous work explored only mono-lingual approaches, and multi-lingual threatening content detection was not studied. This research addressed the task of Multi-lingual Threatening Content Detection (MTCD) in Urdu and English languages by exploiting transfer learning methodology with fine-tuning techniques. To address the multi-lingual task, we investigated two methodologies: 1) Joint multi-lingual, and 2) Joint-translated method. The former approach employs the concept of building a universal classifier for different languages whereas the latter approach applies the translation process to transform the text into one language and then perform classification. We explore the Multilingual Representations for Indian Languages (MuRIL) and Robustly Optimized BERT Pre-Training Approach (RoBERTa) with fine-tuning that already demonstrated state-of-the-art in capturing the contextual and semantic characteristics within the text. For hyper-parameters, manual search and grid search strategies are utilized to find the optimum values. Various experiments are performed on bi-lingual English and Urdu datasets and findings revealed that the proposed methodology outperformed the baselines and showed benchmark performance. The RoBERTa model achieved the highest performance by demonstrating 92% accuracy and 90% macro f1-score with the joint multi-lingual approach.

show abstract

HOTTEST: Hate and Offensive content identification in Tamil using Transformers and Enhanced STemming

Cited by 17 publications

References 7 publications

Taming Toxic Talk: Detecting Offensive Online Content in Hausa

Taming Toxic Talk: Detecting Offensive Online Content in Hausa

SETAR: Stacking Ensemble Learning for Thai Sentiment Analysis Using RoBERTa and Hybrid Feature Representation

Fine-Tuning Transformer Models Using Transfer Learning for Multilingual Threatening Text Identification

Contact Info

Product

Resources

About