NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier

Seganti, Alessandro; Sobol, Helena; Orlova, Iryna; Kim, Hannam; Staniszewski, Jakub; Krumholc, Tymoteusz; Koziel, Krystian

doi:10.18653/v1/s19-2126

Cited by 16 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, many teams used ensembles of deep learning models (Mahata et al, 2019) to benefit from its minimal need for features engineering and ability to boost the classifier performance. Moreover, to address the small dataset problem some teams used Bert model (Liu et al, 2019) and others utilized external datasets to further increase the training data (Seganti et al, 2019). In this paper, we show that using back translation data augmentation and transfer learning significantly improves the offensive language prediction performance.…”

Section: Related Workmentioning

confidence: 84%

AlexU-BackTranslation-TL at SemEval-2020 Task 12: Improving Offensive Language Detection Using Data Augmentation and Transfer Learning

Ibrahim

Torki

El-Makky

2020

Proceedings of the Fourteenth Workshop on Semantic Evaluation

View full text Add to dashboard Cite

Social media platforms, online news commenting spaces, and many other public forums have become widely known for issues of abusive behavior such as cyber-bullying and personal attacks. In this paper, we use the annotated tweets of Offensive Language Identification Dataset (OLID) to train three levels of deep learning classifiers to solve the three sub-tasks associated with the dataset. Sub-task A is to determine if the tweet is toxic or not. Then, for offensive tweets, sub-task B requires determining whether the toxicity is targeted. Finally, for sub-task C, we predict the target of the offense; i.e. a group, individual or other entity. In our solution, we tackle the problem of class imbalance in the dataset by using back translation for data augmentation and utilizing fine-tuned BERT model in an ensemble of deep learning classifiers. We used this solution to participate in the three English sub-tasks of SemEval-2020 task 12. The proposed solution achieved 0.91393, 0.6300 and 0.57607 macro F1-average in sub-tasks A, B and C respectively. We achieved the 8th, 14th and 21st places for sub-tasks A, B and C respectively.

show abstract

Section: Related Workmentioning

confidence: 84%

AlexU-BackTranslation-TL at SemEval-2020 Task 12: Improving Offensive Language Detection Using Data Augmentation and Transfer Learning

Ibrahim

Torki

El-Makky

2020

Proceedings of the Fourteenth Workshop on Semantic Evaluation

View full text Add to dashboard Cite

show abstract

“…It is also possible to use an ensemble of the above methods [59,61]. In fact, such an approach has recently provided the best performance (based on the average performance on all subtasks) in a competition among more than fifty participating teams [77].…”

Section: Related Workmentioning

confidence: 99%

“…As the overall best performing model in task 6 of the 2019 Semeval competition was an ensemble of different models [77], we also experienced an ensemble solution. For this experiment, to still incorporate additional data in RoBERTa, we combined the RoBERTa model trained on HASOC data with the FastText model trained on all databases, using pretrained word embeddings.…”

Section: Ensemble Of Classifiersmentioning

confidence: 99%

Challenges of Hate Speech Detection in Social Media

2021

View full text Add to dashboard Cite

The detection of hate speech in social media is a crucial task. The uncontrolled spread of hate has the potential to gravely damage our society, and severely harm marginalized people or groups. A major arena for spreading hate speech online is social media. This significantly contributes to the difficulty of automatic detection, as social media posts include paralinguistic signals (e.g. emoticons, and hashtags), and their linguistic content contains plenty of poorly written text. Another difficulty is presented by the context-dependent nature of the task, and the lack of consensus on what constitutes as hate speech, which makes the task difficult even for humans. This makes the task of creating large labeled corpora difficult, and resource consuming. The problem posed by ungrammatical text has been largely mitigated by the recent emergence of deep neural network (DNN) architectures that have the capacity to efficiently learn various features. For this reason, we proposed a deep natural language processing (NLP) model—combining convolutional and recurrent layers—for the automatic detection of hate speech in social media data. We have applied our model on the HASOC2019 corpus, and attained a macro F1 score of 0.63 in hate speech detection on the test set of HASOC. The capacity of DNNs for efficient learning, however, also means an increased risk of overfitting. Particularly, with limited training data available (as was the case for HASOC). For this reason, we investigated different methods for expanding resources used. We have explored various opportunities, such as leveraging unlabeled data, similarly labeled corpora, as well as the use of novel models. Our results showed that by doing so, it was possible to significantly increase the classification score attained.

show abstract

“…The systems developed for these tasks used cutting-edge NLP, machine learning and deep learning techniques. Some key systems for OffensEval and HatEval such as Fermi (Indurthi et al, 2019) used Universal Sentence Encoder to build a SVM model, NLPR@SAPOL (Seganti et al, 2019) used an ensemble of deep learning models like OpenAI Finetune and Transformer, while NULI developed a BERT-based model. Although some systems have achieved reasonable performance (e.g., 0.82 F1-score for subtask A of OffensEval by NULI), most other systems still lack 'good' performance for other subtasks such as identifying target and type of offenses.…”

Section: Related Workmentioning

confidence: 99%

AdelaideCyC at SemEval-2020 Task 12: Ensemble of Classifiers for Offensive Language Detection in Social Media

Herath¹,

Atapattu

Dung³

et al. 2020

Proceedings of the Fourteenth Workshop on Semantic Evaluation

View full text Add to dashboard Cite

This paper describes the systems our team (AdelaideCyC) has developed for SemEval Task 12 (OffensEval 2020) to detect offensive language in social media. The challenge focuses on three subtasks -offensive language identification (subtask A), offense type identification (subtask B), and offense target identification (subtask C). Our team has participated in all the three subtasks. We have developed machine learning and deep learning-based ensembles of models. We have achieved F1-scores of 0.906, 0.552, and 0.623 in subtask A, B, and C respectively. While our performance scores are promising for subtask A, the results demonstrate that subtask B and C still remain challenging to classify.

show abstract

NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier

Cited by 16 publications

References 26 publications

AlexU-BackTranslation-TL at SemEval-2020 Task 12: Improving Offensive Language Detection Using Data Augmentation and Transfer Learning

AlexU-BackTranslation-TL at SemEval-2020 Task 12: Improving Offensive Language Detection Using Data Augmentation and Transfer Learning

Challenges of Hate Speech Detection in Social Media

AdelaideCyC at SemEval-2020 Task 12: Ensemble of Classifiers for Offensive Language Detection in Social Media

Contact Info

Product

Resources

About