Emad at SemEval-2019 Task 6: Offensive Language Identification using Traditional Machine Learning and Deep Learning approaches

Kebriaei, Emad; Karimi, Samaneh; Sabri, Nazanin; Shakery, Azadeh

doi:10.18653/v1/s19-2107

Cited by 7 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these three approaches, the rule-based black-list was the best with a 0.73 F-score. Kebriaei et al proposed machine learning algorithms, deep learning algorithms and the ensemble of these algorithms, and also, realized data augmentation from different sources [29]. The most successful Fscore (0.76) was obtained with the application of SVM to expanded data.…”

Section: Related Workmentioning

confidence: 99%

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Ekinci¹

2022

Sakarya University Journal of Computer and Information Sciences

View full text Add to dashboard Cite

The classification of documents is one of the problems studied since ancient times and still continues to be studied. With social media becoming a part of daily life and its misuse, the importance of text classification has started to increase. This paper investigates the effect of data augmentation with sentence generation on classification performance in an imbalanced dataset. We propose an LSTM based sentence generation method, Term Frequency-Inverse Document Frequency (TF-IDF) and Word2vec and apply Logistic Regression (LR), Support Vector Machine (SVM), K Nearest Neighbor (KNN), Multilayer Perceptron (MLP), Extremely Randomized Trees (Extra tree), Random Forest, eXtreme Gradient Boosting (Xgboost), Adaptive Boosting (AdaBoost) and Bagging. Our experiment results on an imbalanced Offensive Language Identification Dataset (OLID) that machine learning with sentence generation significantly outperforms.

show abstract

Section: Related Workmentioning

confidence: 99%

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Ekinci¹

2022

Sakarya University Journal of Computer and Information Sciences

View full text Add to dashboard Cite

show abstract

“…In [44], they applied dependency based embeddings for word substitution to generate text while leveraging textual membership queries. Some other studies leveraged solutions such as shifting the position of words in a zero padded representation [30], synthetic minority oversampling, random over-and under-sampling, and AdaSYN [29], adding common misspelling of words to data and collecting tweets that contain swear words in conjunction with positive adjectives or racial and religious tweets [38], adding tweets with disgust and anger emotions from suspended accounts to the data [1], bootstrapping from another dataset; embedding based [13] or sentiment polarity based [8] methods.…”

Section: Related Studiesmentioning

confidence: 99%

“…The research area of hate speech detection framed as a text classification task has come a long way in recent years. The outpour of research studies tackle different aspects of the problem and the resulting proposed solutions include the manual extraction of generic and task-specific features and the careful design of machine learning architectures [13,20,21,23,27]. Since most of the existing work have formulated the problem of detecting textual hate speech as a text classification task, it naturally requires access to a large source of clean, bias free, balanced textual data.…”

Section: Introductionmentioning

confidence: 99%

Token replacement-based data augmentation methods for hate speech detection

2022

View full text Add to dashboard Cite

Hate speech detection mostly involves the use of text data. This data, usually sourced from various social media platforms, have been known to be plagued with numerous issues that result in a reduction of its quality and hence, the quality of the trained models. Some of these issues are the lack of diversity and the diminutive class of interest in the dataset which results in overfitted models that do not generalize well on other or newly collected data. The different ways of handling these issues include augmenting the data with diverse samples, engineering non-redundant features or designing robust classification models. In this study, the focus is on the data augmentation aspect. Data augmentation is a popular method for improving the quality of existing datasets by generating synthetic samples that mimic the distribution of the original samples. There is a lack of extensive studies on how hate speech texts respond to varying textual data augmentation techniques and methods. Specifically, we provide further insight into the token replacement method of textual data augmentation by performing empirical studies that investigate which embedding method(s) is a robust source of synonym for replacement process, what effective method(s) can be used to select words to be replaced, and how to confirm if the label within each class is preserved. Our proposed methods, validated on two commonly used hate speech datasets affected by a known lack of diversity and diminutive class of interest issues, significantly improve classification performance and provides insights into token replacement methods.

show abstract

“…Traditional machine learning (ML) approaches such as support vector machines (SVM) by Perelló et al (2019) can be trained on hate speech tweets by identifying n-grams features which could be improved further by combining word embedding with sentiment features. Research by Kebriaei et al (2019) shows how a convolutional neural network (CNN) shows higher macro averaged F1-score than traditional ML approaches such as SVM, random forest (RF) and naive Bayes (NB). (Rajendran et al, 2019) uses an ensemble of classifiers to classify the offensive text in an imbalanced dataset by using models with Synthetic Minority Over-sampling technique (SMOTE).…”

Section: Related Workmentioning

confidence: 99%

NUIG at SemEval-2020 Task 12: Pseudo Labelling for Offensive Content Classification

Suryawanshi¹,

Arčan²,

Buitelaar³

2020

Proceedings of the Fourteenth Workshop on Semantic Evaluation

View full text Add to dashboard Cite

This work addresses the classification problem defined by sub-task A (English only) of the OffensEval 2020 challenge. We used a semi-supervised approach to classify given tweets into an offensive (OFF) or not-offensive (NOT) class. As the OffensEval 2020 dataset is loosely labelled with confidence scores given by unsupervised models, we used last year's offensive language identification dataset (OLID) to label the OffensEval 2020 dataset. Our approach uses a pseudo-labelling method to annotate the current dataset. We trained four text classifiers on the OLID dataset and the classifier with the highest macro-averaged F1-score has been used to pseudo label the OffensEval 2020 dataset. The same model which performed best amongst four text classifiers on OLID dataset has been trained on the combined dataset of OLID and pseudo labelled OffensEval 2020. We evaluated the classifiers with precision, recall and macro-averaged F1-score as the primary evaluation metric on the OLID and OffensEval 2020 datasets.

show abstract

Emad at SemEval-2019 Task 6: Offensive Language Identification using Traditional Machine Learning and Deep Learning approaches

Cited by 7 publications

References 8 publications

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Classification of Imbalanced Offensive Dataset – Sentence Generation for Minority Class with LSTM

Token replacement-based data augmentation methods for hate speech detection

NUIG at SemEval-2020 Task 12: Pseudo Labelling for Offensive Content Classification

Contact Info

Product

Resources

About